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Preface 


The  purpose  of  this  thesis  was  to  discover  to  what  extent  theoretical  models  are 
used  to  establish  cost  and  schedule  variance  thresholds  on  the  Department  of  Defense 
major  program  contracts.  Further,  it  identifies  how  cost  and  schedule  variance 
thresholds  are  actually  established  for  these  contracts.  The  authors  recognized  that  the 
number  of  our  interviews  would  be  limited  due  to  funding  and  time  constraints. 
However,  this  research  provided  a  general  understanding  of  the  variance  threshold 
concept  and  its  possible  application  to  program  management.  It  could  also  serve  as  a 
starting  point  for  an  in-depth  study  of  the  subject. 
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and  guidance  and  providing  many  points  of  contacts  for  our  interviews  throughout  the 
program.  We  also  wish  to  thank  the  entire  professional  staff  of  the  AFIT  Library  for 
the  excellent  services  in  providing  modem  information  retrieval  systems  and  locating 
necessary  materials  for  this  thesis.  We  would  like  to  recognize  the  tremendous 
professional  support  and  enthusiasm  from  all  interview  participants.  Finally,  the 
authors  would  like  to  thank  Lily  T.  Nguyen  whose  patience,  understanding,  and  words 
of  encouragement  contributed  much  to  the  success  of  this  learning  process. 

Tuan  A.  Hoang 
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Abstra 

This  thesis  examined  to  what  extent  the  i^tical  threshold  models  are  used  to 
establish  cost  and  schedule  variance  threshold,,  <an  Depanment  of  Defense  (DoD) 
major  program  contracts  and  identified  how  tlase  thresholds  are  actually  established. 
First,  numerous  theoretical  methods  for  establc<  .^ing  variance  thresholds  including  the 
accounting  approach,  the  statistical  process  co/,-rol  approach,  and  the  Dyckman  and 
Kaplan  models  were  investigated  and  assessed.  Interviews  were  then  conducted  with  a 
sample  of  DoD  and  civilian  defense  contracu  r  personnel  who  had  first-hand 
knowledge  of  the  establishment  of  cost  and  sg^dule  variance  thresholds.  Findings 
indicated  that  none  of  these  methods  was  beitv^  used.  Repeatedly,  interviewees  related 
that  threshold  levels  for  new  contracts  were  e  ^1<d)lished  by  either  management 
experience  and  judgment  or  by  copying  threshold  levels  from  previous  contracts.  One 
possible  explanation  for  this  occurrence  is  the  a  pparent  lack  of  awareness  of  the  many 
theoretical  models  available.  This  lack  of  knowledge  could  have  been  the  result  of 
inadequate  supplemental  training  which  could  expose  personnel  to  these  additional 
methods.  In  addition,  neither  government  nor  c/*fense  contractor  documents  provided 
any  specific  methods  or  techniques  to  derive  -tKrosholds.  This  lack  of  specific 
guidance  further  contributes  to  the  necessary  rd  iance  on  personal  experience  and 
judgment  to  set  the  threshold  levels.  Upper  n'taflagement  should  provide  functional 
personnel  with  additional  supplemental  trainiift<5to  expose  them  to  the  various 
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theoretical  methods. 


A  STUDY  OF  THE  ESTABLISHMENT  OF  COST 
AND  SCHEDULE  VARIANCE  THRESHOLDS 
ON  DEPARTMENT  OF  DEFENSE 
MAJOR  PROGRAM  CONTRACTS 

I.  Introduction 

General  Issues 

Since  the  mid-1980s  there  has  been  an  increasing  perception  of 
mismanagement,  waste,  and  even  fraud  and  abuse  in  defense  acquisition  programs. 
Critics  have  pointed  to  the  Department  of  Defense  (DoD)  paying  $'5000  for  an 
ordinary  hammer  and  $2000  for  a  plastic  cap  for  a  stool  leg  (Gansler,  1991:4).  This  is 
illustrated  by  the  headlines  in  a  major  American  newspaper: 

"Arms  Systems  Running  Far  Over  Budget" 

The  pentagon  is  seeking  large  funding  increases  for  20  major  weapons 
programs,  many  of  which  have  had  tec'^nical  problems  and  huge  cost 
overruns,  according  to  the  draft  of  a  General  Account  Office  report. 

.  .  .it  depicts  significant  problems  in  .some  of  the  nation’s  most 
important  weapons  programs  just  as  they  are  to  enter  full-scale 
development  or  production,  milestones  that  will  require  sharp  funding 
increases. 

The  report  that  was  leaked  to  the  news  media,  indicates  that  overall  the 
20  weapons  systems  have  experienced  cost  increa.ses  of  $20.3  billion. 
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Thirteen  of  the  20  programs  are  behind  schedule,  nine  have  posted  cost 

increases.  .  .  (Vartabedian,  1987:1-1) 

In  another  program.  Congressional  scrutiny  of  the  United  States  Air  Force’s 
C-17  program  widened  as  internal  DoD  documents  being  circulated  in  Congress 
showed  that  McDonnell  Douglas  is  likely  to  exceed  the  development  program  cost 
ceiling  by  $1. 4-2.6  billion.  An  Office  of  the  Secretary  of  Defense  analysis  concluded 
it  will  cost  $8-9.2  billion  to  complete  the  full-scale  development  contract  and  the  first 
six  production  aircraft.  This  range  is  far  higher  than  the  USAF’s  recently  revised 
figure  of  $7.3  billion,  which  would  result  in  an  overrun  of  $700  million.  McDonnell 
Douglas  estimates  it  will  cost  $7  billion  or  $390  million  above  the  $6.6-billion  ceiling. 
Representative  John  Dingell  (D.-Mi..h.),  chairman  of  the  House  Energy  and  Commerce 
Oversight  and  Investigations  Subcommittee,  told  Defense  Secretary  Richard  B.  Cheney 
in  an  August  26  letter  that  the  C-17  contract  should  be  canceled  for  default  (Gilmartin, 
1991:25). 

Another  scenario  is  the  ill-fated  A- 1 2  program,  shot  down  by  the  Secretary  of 
Defense,  Richard  Cheney.  Originally,  the  Department  of  the  Navy  expected  to 
produce  620  high-tech  stealth  aircraft  at  a  cost  of  $57  billion.  The  program  was 
running  $2.7  billion  over  its  fixed  price  contract  cost  of  $4.8  billion  for  development. 

It  was  also  18  months  behind  schedule.  The  bottom  line  was  that  no  one  in  the 
Department  of  the  Navy  was  able  to  tell  the  Secretary  Cheney  how  much  money  it 
would  take  to  finish  the  development  program  (Magnuson,  1991:46). 

These  examples  paint  a  very  bleak  picture  of  DoD  management.  However,  in 
comparison  to  many  other  organizations,  the  DoD  does  a  'relatively "  go<xl  job  of 
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controlling  cost  overruns  (Figure  1).  But  "relatively”  may  not  be  good  enough. 

Traditionally,  acquisition  cycle  cost  overruns  on  DoD  procurement  programs  have 

been  between  40  and  100  percent  (Gansler,  1991:171).  Table  1  depicts  procurement 

plans  versus  actual  results  for  selected  weapons  systems  between  1981  and  1985 

(Gansler,  1991:132).  If  the  trend  continues,  by  the  year  2054  the  U.S.  will  be  able  to 

purchase  a  single  fighter  plane  per  year  (Augustine,  1983:55). 

However,  these  perceived  problems  should  be  put  in  perspective.  The  DoD’s 

seemingly  uncontrollable  cost  overrun  problems  may  be  due  to  the  enormous  size  of 

the  organization  and  the  large  volume  of  contract  transactions.  As  Gansler  points  out: 

It  is  not  an  exaggeration  to  state  that  defense  acquisition  is  the  largest 
business  enterprise  in  the  free  world.  Annual  purchases  by  the 
Department  of  Defense  total  around  170  billion  dollars  (more  than  the 
combined  purchases  of  General  Motors,  Exxon,  and  IBM).  The  DoD’s 
research  and  development  expenditures  alone  are  7.5  times  the 
combined  R&D  expenditures  of  France,  Germany,  the  United  Kingdom, 
and  Japan.  Defense  acquisition  involves  almost  15  million  separate 
contract  actions  each  year,  implemented  through  over  a  thousand  buying 
offices  around  the  world.  More  than  300,000  industrial  suppliers  are 
involved,  and  about  4.5  million  different  kinds  of  items  are  purchased 
each  year.  (Gansler,  1991:142) 

Considering  the  magnitude  of  the  v'orkload  it  is  almost  certain  that  occasional  errors 
will  occur.  Even  a  level  of  99.99%  perfect  transactions  would  result  in  1,500  errors 
each  year  (Gansler,  1991:5).  However,  in  order  to  minimize  the  number  of  these 
occurrences.  DoD  implemented  a  system  to  monitor  the  acquisition  process. 

In  1967  the  Department  of  Defense  implemented  the  Cost/Schedule  Control 
Systems  Criteria  (C/SCSC)  to  standardize  cost  and  schedule  reporting  requirements 
and  to  provide  visibility  of  acquisition  program  accomplishments.  C/SCSC  is  required 
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Figure  1.  Cost  Growth  of  Major  FYograms 
(Adapted  from  Gansler,  1991:5) 
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ABLE  1 


PROCUREMENT  PLAN  VERSUS  ACTUAL  RESULTS 
FOR  SELECTED  WEAPONS  SYSTEMS,  1981-1985 
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TEM 
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PERCENTAGE  CHANGE 
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6332 

1.63 
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8966 

2.36 
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42 

45 
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3720 

3591 

0.97 

2855 

4522 

1.58 

-23 

26 

64 

AH-64 

248 

2615 

9.20 

315 

3955 

12.56 

11 

51 

36 

F/A-18 

656 

13692 

20.90 

375 

12387 

33.00 

-43 

-10 

58 

F-15 

90 

2764 

30.70 

195 

7379 

37  80 

117 

167 

23 

F-16 

660 

8717 

13.20 

714 

11713 

16.40 

8 

34 

24 

a.  As  contained  in  FY81  budget  and  five-year  plan.  (Adapted  from  Gansicr,  1991: 132) 

b.  In  constant  1985  dollars. 
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for  major  acquisition  programs  exceeding  $250  million  for  procurement  contracts  and 
$60  million  for  research  and  development  efforts. 

The  C/SCSC  consists  of  35  criteria  which  specify  the  minimum  requirements  a 
contractor’s  management  control  system  must  meet.  These  criteria  are  still  in  effect 
some  25  years  later  and  have  been  adopted  by  other  United  States  government 
agencies,  such  as  the  Department  of  Energy  (DoE)  and  the  National  Aeronautics  and 
Space  Administration  (NASA)  as  well  as  the  North  American  Treaty  Organization 
(NATO)  and  the  Swedish  military.  Despite  this  widespread  international  recognition, 
DoE  officially  abandoned  the  C/SCSC  in  August  of  1992  after  having  used  them  to 
help  manage  their  programs  for  20  years  (Fleming,  1993:6). 

The  major  objectives  of  the  C/SCSC  are: 

For  contractors  to  use  effective  internal  cost  and  schedule  management 
control  systems,  and 

For  the  government  to  be  able  to  rely  on  timely  and  auditable  data 
produced  by  those  systems  for  determining  product-oriented  contract 
status.  (Fleming,  1992:25) 

More  specifically,  the  C/SCSC  was  developed  to: 

Provide  the  contractor  and  the  government  program  office  managers 
with  accurate  data  to  monitor  execution  of  their  program; 

Provide  an  adequate  basis  for  responsible  decision  making  by  both 
contractor  management  and  DoD  Component  personnel,  by  requiring 
that  contractors’  internal  management  control  systems  produce  data  that: 

(a)  indicate  work  progress;  (b)  properly  relate  cost,  schedule  and 
technical  accomplishment;  (c)  are  valid,  timely  and  able  to  be  audited; 
and  (d)  supply  DoD  Component  managers  with  information  at  a 
practical  level  of  summarization;  and 

Bring  to  the  attention  of  DoD  contractors,  and  encourage  them  to  accept 
and  install,  management  control  systems  and  procedures  that  are  most 
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effective  in  meeting  requirements  and  controlling  contract  performance.  (DoDI 
5000.2,  Part  11,  Section  B,  paragraph  1) 

An  important  aspect  of  the  35  criteria  is  performance  measurement  to  analyze 
departures  from  the  planned,  budgeted,  or  expected  performance.  A  depanure  from 
the  planned,  budgeted  or  expected  performance  is  known  as  a  "variance."  Actual 
performance  and  planned  performance  rarely  coincide.  As  a  result,  variances  are 
expected.  However,  when  does  a  variance  become  significant  enough  to  require 
management  attention? 

One  way  to  address  this  problem  is  by  using  control  limits  called  "variance 

thresholds"  (Figure  2).  When  a  variance  exceeds  these  thresholds,  it  is  considered  a 

significant  variance,  and  an  analysis  of  the  variance  is  required.  The  key  is  to  set 

variance  thresholds  at  appropriate  levels  so  that  only  significant  variances  are  detected 

and  actions  be  taken  to  correct  the  problem.  Variances  that  do  not  exceed  a  threshold 

do  not  require  investigation.  Investigation  of  insignificant  variances  which  have  little 

potential  for  adverse  impact  on  a  program  is  both  costly  and  time  consuming. 

Careful  selection  of  these  thresholds  is  necessary  to  prevent  unnecessary 
work  associated  with  preparing  an  excessive  number  of  written  analyses. 

The  analysis  of  every  cost  and  schedule  variance  is  usually  unnecessary 
and  unproductive.  (DoD,  1987:3-17) 

Problem  Statement 

Despite  the  emphasis  for  appropriate  levels  of  variance  thresholds  cited  in  the 
Joint  Implementation  Guide  (JIG),  these  levels  mean  different  things  to  different 
people.  For  instance,  a  significant  variance  threshold  in  a  relatively  low  dollar  value 
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Figure  2.  Graphic  Presentation  of  Variance  Thresholds 
(Adapted  from  Nicholas,  1990:396) 


8 


program  may  not  be  viewed  as  significant  in  a  relatively  high  dollar  value  program. 

In  1984,  a  study  by  Arthur  D.  Little  Company  attempted  to  determine  the  utility  of 
C/SCSC.  This  study  surveyed  12  defense  contractors  and  12  DoD  program  offices, 
four  each  from  the  Air  Force,  Army,  and  Navy.  One  important  finding  with  regard  to 
Cost/Schedule  Performance  Reponing  (CPR)  emphasizes  the  contrasting  perception 
between  government  personnel  and  defense  contractors  concerning  CPR  reporting 
requirements. 

Government  personnel  were  more  concerned  with  the  timeliness  and  the  quality 
of  the  analysis  reports  than  the  level  of  variance  thresholds.  Government  personnel 
felt  that  the  CPR  variances  thresholds  were  not  as  critical  as  the  quality  of  variance 
reporting  in  CPRs.  In  general,  government  personnel  felt  the  quality  of  variance 
reponing  was  poor,  particularly  the  analysis  of  schedule  variances  (Little,  1984:111-25). 

On  the  other  hand,  contractors  were  critical  of  excessive  reporting  requirements 
due  to  overly  stringent  variance  thresholds  required  by  the  government.  Contractors 
felt  variance  analyses  were  required  too  frequently,  leaving  little  time  to  perform  the 
contract  work.  Contractors  consistently  wanted  less  variance  reporting  (Little, 
1984:111-24). 

The  problem  was  also  addressed  in  an  October  1979  study.  The  National 
Security  Industrial  Association  Management  Systems  Subcommittee  prepared  the 
Cost/Schedule  Systems  Compendium.  This  compendium  surveyed  companies  which 
had  been  identified  as  having  experience  with  performance  measurement  requirements. 
The  purpose  of  this  survey  was  to  examine  the  responses  and  provide 
recommendations  for  improving  the  requirements  and  implementation  of  C/SCSC. 
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From  the  74  responses  received,  the  results  pertaining  to  the  use  of  variance  thresholds 
indicate: 

Most  contracts  contain  a  percentage  figure  for  current  month 
cumulative-to-date  and  at  completion  periods.  Many  also  contain  a 
minimum  and/or  maximum  dollar  threshold  intended  to  truncate 
extremes  caused  by  percentages  applied  to  high  and  low  value  Work 
Breakdown  Structure  (WBS)  item.  (NSIA,  1980:1-7) 

For  example,  39  percent  of  the  responses  required  the  same  threshold  to 
be  applied  to  current  period  and  cum-to-date  variances.  Fony-seven 
percent  of  the  responses  required  the  same  threshold  applied  to  cum-to- 
date  and  at  completion  variances.  (NSIA,  1980:1-14) 


Specific  Problem 

Different  interpretations  of  variance  thresholds  between  government  and 
defense  contractor  personnel  indicate  the  need  for  setting  appropriate  variance 
thresholds.  Recognizing  the  importance  of  appropriate  threshold  levels,  researchers 
have  developed  many  theoretical  models  to  help  establish  such  threshold  levels  (e.g., 
Dyckman  (1969)  and  Kaplan  (1975)).  Although  variance  thresholds  are  found  in 
many  defense  acquisition  contracts,  little  is  known  about  the  methods  the  government 
and  contractors  use  to  establish  these  threshold  values. 

The  research  objectives  of  this  thesis  are: 

1)  to  discover  to  what  extent  these  theoretical  models  are  used  to  establish  cost 

and  schedule  variance  thresholds  on  DoD  major  program  contracts,  and 

2)  to  identify  how  cost  and  schedule  variance  thresholds  are  established  on 

DoD  major  program  contracts. 
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Investigative  Questions 


In  order  to  achieve  these  objectives,  the  following  investigative  questions  must 
be  addressed: 

1)  What  models  are  available  for  establishing  variance  thresholds? 

2)  Are  any  of  these  models  applicable  to  DoD  major  program  contracts? 

3)  Who  is  responsible  for  setting  cost  and  schedule  variance  thresholds  on  DoD 
major  program  contracts? 

4)  Are  DoD  personnel  aware  of  these  models? 

5)  Are  any  of  these  models  being  used  by  the  DoD? 

6)  How  are  cost  and  schedule  variance  thresholds  established  on  DoD  major 
program  contracts? 

7)  What  models  for  establishing  cost  and  schedule  variance  thresholds  are 

used? 

8)  Do  the  chosen  methods  vary  depending  on  program  type,  preparation  time, 
risk,  contract  phase,  and  contract  cost? 

The  first  two  investigative  questions  will  require  a  thorough  literature  review 
on  the  topic  of  variance  thresholds.  The  advantages,  disadvantages,  differences,  and 
the  applicability  of  these  models  to  DoD  major  program  contracts  will  be  described. 
The  remainder  of  the  investigative  questions  will  require  personal  interviews  with 
government  system  program  office  and  contractor  personnel. 
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Scope  and  Limitations 


This  thesis  is  a  formalized  study  consisting  of  eight  structured  investigative 
questions.  It  is  descriptive  in  nature.  It  examines  things  as  they  exist  and  does  not 
attempt  to  manipulate  any  variables. 

The  population  of  interest  includes  all  government  financial  managers  and 
civilian  contractor  counterparts  who  establish  cost  and  schedule  variance  thresholds  for 
DoD  major  program  contracts.  The  interviews  will  be  conducted  with  a  sample  of 
Aeronautical  System  Center  (ASC)  staff  and  System  Program  Office  (SPO)  personnel 
at  Wright-Patterson  Air  Force  Base,  Ohio  who  have  first  hand  knowledge  of  the 
establishment  of  variance  thresholds.  Additionally,  several  contractor  personnel  will 
be  interviewed. 

As  this  thesis  is  descriptive  in  nature,  rigorous  statistical  methods  to  manipulate 
data  will  not  be  used.  This  limits  the  inferences  which  can  be  drawn  concerning  the 
population.  However,  basic  descriptive  statistics  of  data  will  be  calculated  and 
presented. 

Summary  and  Overview 

Cost  and  schedule  overruns  are  a  source  of  inspiration  for  critics  of  the  DoD. 
Due  to  increasingly  limited  resources  all  measures  for  controlling  cost  must  be 
exercised. 

In  1967  the  DoD  implemented  C/SCSC  to  standardize  cost  and  .schedule 
reporting  requirements  and  to  provide  visibility  of  acquisition  program 
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accomplishments.  The  C/SCSC  Analysis  Group  addresses  this  problem  through  the 
use  of  variance  thresholds. 

The  JIG  specifically  discusses  the  importance  of  establishing  appropriate  levels 
of  cost  and  schedule  variance  thresholds.  However,  despite  the  emphasis  on 
appropriate  levels  of  variance  thresholds  cited  in  the  JIG,  these  levels  mean  different 
things  to  government  and  contractor  personnel. 

The  purpose  of  this  thesis  is  to  examine  to  what  extent  theoretical  threshold 
models  are  used  to  establish  cost  and  schedule  variance  thresholds  on  DoD  major 
program  contracts,  to  identify  how  these  thresholds  are  established,  and  to  report  those 
findings. 

The  first  two  investigative  questions  will  require  a  thorough  literature  review 
on  the  topic  of  variance  thresholds.  The  remainder  of  the  investigative  questions  will 
require  personal  interviews. 

After  25  years,  C/SCSC  has  been  proven  as  a  good  management  tool  for 
controlling  cost  and  schedule  on  major  DoD  programs.  However,  problems  still  exist. 
Numerous  articles  concerning  cost  overruns  and  schedule  delays  seem  to  indicate 
serious  lack  of  attention  by  program  management.  Different  interpretations  of 
variance  thresholds  between  government  and  contractor  personnel  indicate  the  need  for 
setting  appropriate  variance  thresholds.  If  C/SCSC  is  to  be  an  effective  management 
tool,  then  proper  attention  should  be  given  to  the  establishment  of  appropriate  cost  and 
schedule  variance  thresholds. 

The  next  chapter  examines  a  number  of  theoretical  models  which  can  be  used 
to  establish  cost  and  schedule  variance  thresholds.  It  de.scribes  the  advantages, 
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disadvantages,  and  the  applicability  of  these  models  to  DoD  major  program  contracts. 
In  addition,  contractor  and  government  documents  for  establishing  variance  thresholds 
are  reviewed. 

The  chapter  on  methodology  describes  the  process  used  to  acquire  data  to 
answer  the  six  investigative  questions.  In  the  subsequent  chapter,  data  are  analyzed 
and  presented.  Finally,  conclusions  and  recommendations  are  presented. 
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//.  Literature  Review 


Introduction 

The  previous  chapter  indicated  the  need  for  setting  appropriate  variance 
thresholds.  There  are  many  methods  suggested  in  literature  which  are  used  to 
establish  variance  thresholds  and  may  be  applicable  to  government  contract 
management. 

This  literature  review  answers  the  first  two  investigative  questions  addressed  in 
the  introduction  chapter.  These  questions  are: 

1)  What  models  are  available  for  establishing  cost  and  schedule  variance 
thresholds? 

2)  Are  any  of  these  models  applicable  to  DoD  major  program  contracts? 

This  chapter  examines  a  number  of  theoretical  models  which  may  be  used  to 

establish  cost  and  schedule  variance  thresholds.  These  methods  range  from  relatively 
simple  accounting  approaches  to  more  complex  statistical  process  control  methods. 

First,  this  chapter  examines  the  basic  control  aspects  of  three  quantitative 
models  for  accounting  control:  1)  the  traditional  accounting  model  using  standard 
costing,  2)  an  accounting  model  based  on  classical  statistical  theory,  and  3)  a  control 
model  based  on  modem  decision  theory.  Second,  a  variety  of  statistical  process 
control  models  such  as  Shewhart  Bar  Chart,  Cumulative  Sum  Chart,  Economic  X-Bar 
Chart,  and  Economic  Cusum  Chart  are  examined.  Finally,  threshold  models  developed 
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by  T.R.  Dyckman  and  Robert  Kaplan  will  be  described.  Table  2  outlines  the  models 
examined  in  this  chapter. 

For  each  of  these  models  the  discussion  includes  the  advantages,  disadvantages, 
and  the  applicability  of  the  model  to  DoD  major  program  contracts. 

Accounting  Approach 

Accounting  control  is  a  management  function  concerned  with  discovering 
deviations  from  planned  activities  and  prompt  correction  of  these  deviations  (Onsi, 
1967:321).  Onsi  examined  three  quantitative- models  for  accounting  control  which 
may  be  used  to  identify  deviations  and  determine  whether  corrective  actions  are 
necessary.  The  models  examined  are:  1)  the  traditional  accounting  model  using 
standard  costing,  2)  an  accounting  control  model  based  on  classical  statistical  theory, 
and  3)  an  accounting  control  model  based  on  mo<iem  decision  theory. 

Tra'  ,^onal  Accounting  Model.  The  traditional  accounting  model  uses  a 
relatively  informal  method  to  establish  cost  and  schedule  variance  thresholds  (Onsi, 
1967:321).  In  this  model,  management  Judgment  and  experience  are  the  criteria  for 
determining  whether  to  investigate  deviation  from  planned  activities.  Threshold  limits 
are  based  on  the  absolute  or  relative  size  of  the  deviation.  For  example,  a  deviation  of 
less  than  five  percent  may  be  considered  acceptable  and  hence  does  not  require 
investigation.  Such  a  percentage  is  set  based  on  prior  knowledge  of  experienced 
managers. 
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TABLE  2 


SUMMARY  OF  MODELS  EXAMINED 


Accounting  Approach 

Traditional  Accounting  Model 

Accounting  Model  Based  on  Classical  Statistical  Theory 
Accounting  Control  Model  Based  on  Modern  Decision  Theory 
Statistical  Process  Control  Approach 

X-Bar  Chart,  R  Chan,  and  S  Chan 
Cumulative  Sum  Chan 
Economic  X-Bar  Chart 
Economic  Cusum  Chart 
Dyckman  Model 
Kaplan  Model 
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The  major  advantage  of  this  model  is  it  simplicity.  It  is  simple  to  use  because 
no  extensive  or  time-consuming  computations  are  necessary.  Only  managerial 
Judgment  and  experience  are  required.  However,  one  limitation  is  that  it  does  not 
provide  a  clear  indication  when  these  informal  limits  are  no  longer  valid.  For 
instance,  how  do  decision  makers  decide  that  a  five  percent  deviation  from  planned 
activity  is  still  acceptable? 

This  model  can  be  used  to  establish  cost  and  schedule  variance  thresholds  on 
DoD  major  program  contracts.  However,  there  are  potential  drawbacks.  Though 
management  judgment  and  experience  are  beneficial  for  program  control,  often  in  the 
DoD  environment,  managers  do  not  remain  with  the  same  program  throughout  its  life. 
In  addition,  experience  and  judgment  vary  among  managers.  This  may  cause  the  level 
of  program  control  to  be  less  effective.  The  accounting  models  based  on  classical 
statistical  theory  and  modem  decision  theory  evolved  to  address  these  limitations. 

Accounting  Model  Based  on  Classical  Statistical  Theory.  In  contrast  to  the 
traditional  accounting  model,  this  accounting  model  relies  on  classical  statistical  theory 
rather  than  management  judgment  and  experience  to  set  cost  and  .schedule  variance 
thresholds.  In  this  model,  two  types  of  deviations  or  variations  are  possible:  1)  a 
"chance  variation"  which  occurs  randomly  and  2)  an  "assignable  variation"  attributable 
to  systematic  causes  (Onsi,  1967:322).  Chance  variations  are  always  present  as  a 
natural  part  of  the  process  whereas  assignable  variations  arise  from  external  sources 
which  are  not  inherent  to  the  process  (Evans,  1993:206).  Since  chance  variations  are 
random  in  nature,  they  should  not  be  investigated,  in  contrast,  assignable  viuiations 
should  be  investigated  (Onsi,  1967:322).  This  model  makes  four  major  assumptions. 
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First,  expected  cost  is  equal  to  the  mean  of  a  normal  probability  distribution 
(Figure  3).  By  assuming  that  deviations  around  the  mean  are  normally  distributed, 
management  hypothesizes  that  favorable  and  unfavorable  deviations  due  to  random 
causes  will  fall  equally  on  either  side  of  the  standard. 

The  second  assumption  is  that  standards  are  developed  as  ranges,  not  as  point- 
estimates  (such  as,  an  absolute  value  established  in  the  traditional  accounting  model). 
Statistical  analysis  allows  establishing  the  range  of  variations  within  which  deviations 
are  attributed  to  chance.  Variance  thresholds  are  then  set  so  that  chance  and 
assignable  variations  are  identified.  This  analysis  requires  that  both  the  mean  of  the 
distribution  and  its  standard  deviation  be  known.  These  parameters  are  estimated  from 
historical  data  to  represent  the  current  situation. 

Third,  variations  are  investigated  when  one  or  more  consecutive  observations 
lie  outside  the  variance  thresholds  as  illustrated  in  Figure  4. 

Fourth,  the  allowable  deviation  is  represented  by  the  size  of  the  variance 
thresholds.  Determination  of  the  upper  and  lower  control  limits  for  variance  analysis 
depends  upon  the  relative  weight  assigned  to  two  types  of  possible  error.  The  first 
error.  Type  I,  is  the  error  of  investigating  when  it  is  unnecessary;  that  is,  investigating 
a  deviation  which  is  due  to  random  influences.  The  second.  Type  II,  is  the  error  of 
failing  to  investigate  when  there  is  in  fact  an  assignable  variation  from  the  expected 
standard. 

If  the  cost  of  variance  analj  is  is  high  relative  to  the  risk  of  cost  and  schedule 
overruns,  variance  thresholds  should  be  relatively  large.  On  the  other  hand,  if  the  risk 
of  cost  and  schedule  overruns  is  high  relative  to  the  cost  of  variance  analysis,  variance 
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Figure  3.  Normal  Probability  Distribution 
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thresholds  should  be  tight.  In  setting  variance  thresholds,  one  tries  to  ntinimize  the 
penalties  from  an  erroneous  decision  (i.e..  Type  I  or  Type  II  error).  Ideally,  it  is 
desirable  to  balance  the  two  types  of  errors  with  the  degree  of  risk  prevailing  in 
certain  situations  (Onsi,  1967:323). 

However,  this  model  has  some  drawbacks.  The  decision  making  process 
requires  objective  evidence,  given  a  certain  accepted  pre-specified  risk  or  error. 
However,  objective  evidence  may  be  difficult  or  expensive  to  obtain  prior  to  making  a 
decision.  Onsi  also  observed: 

It  does  not  make  an  explicit  structural  use  of  prior  information  and  the  a  priori 
probability  of  the  unknown  parameter.  A  decision  should  be  made  using  both 
prior  information  and  current  objective  evidence.  It  does  not  make  formal  use 
of  the  risks  of  error  of  each  decision  rule  as  a  function  of  the  possible  values 
of  the  parameter  or  standard.  (Onsi,  1967:324) 

This  means  a  reasonable  manager  will  insist  on  a  higher  level  of  significance 
before  rejecting,  on  the  basis  of  given  sample  evidence,  a  strongly  held  belief  as 
compared  with  a  weak  conjecture.  An  analyst  is  more  likely  to  make  an  erroneous 
decision  by  failing  to  formalize  this  information  (Hirshleifer,  1961:477).  A  model 
based  on  modem  decision  theory  will  provide  a  solution  to  some  of  these  problems. 

Accounting  Control  Model  Based  on  Modern  Decision  Theory.  Unlike  the 
traditional  accounting  model  and  the  accounting  model  based  on  classical  statistics,  a 
control  model  based  on  Bayes^ar  modem  decision  theory  will  (Onsi,  1967:324): 

1)  Be  based  on  sample  evidence  and  considers  both  economic  loss  and  prior 
belief  Economic  loss  results  from  either  investigating  a  deviation  when  it  is 
unnecessary  (i.e.,  Type  I  error)  or  not  investigating  a  deviation  when  necessary  (i.e.. 
Type  II  error). 
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2)  Consider  the  prior  state  or  a  priori  probability  distribution  of  the  process. 

3)  Not  require  the  determination  of  Type  I  and  Type  II  errors  as  in  the  model 
based  on  classical  statistical  theory.  However,  the  economic  impact  of  these  errors  is 
incorporated. 

4)  Provide  a  systematic  approach  for  selecting  an  optimum  sample  size.  The 
trade-off  between  the  sampling  cost  and  the  reduction  of  risk  determines  optimum 
sample  size. 

5)  Make  explicit  use  of  the  opportunity  cost  concept  to  evaluate  the  worth  of 
each  action  relative  to  the  best  possible  action  for  the  given  situation.  The  opportunity 
loss  of  any  alternative  is  equal  to  the  difference  between  the  cost  of  that  alternative 
and  the  cost  of  the  alternative  that  would  be  the  best  possible  considering  the  value  of 
the  deviation.  This  model  provides  a  procedure  to  select  the  best  alternative  to 
minimize  expected  economic  loss  of  the  unchosen  alternatives. 

The  model  makes  two  basic  assumptions.  First,  the  manager  decides  whether 
to  investigate  variations  based  on  incomplete  information  which  is  obtained  by 
periodic  random  sampling  of  output.  Any  decision  based  on  incomplete  information 
carries  a  degree  of  risk.  To  minimize  this  type  of  risk,  an  analytic  tool  is  required. 

Second,  the  manager  is  not  only  interested  in  the  cause  of  variation,  but  also 
wishes  to  determine  if  the  process  is  stable.  If  the  process  is  considered  stable,  the 
central  tendency  and  variations  are  expected  to  be  within  a  pre-determined  range. 

The  manager  may  decide  to  investigate  a  deviation  based  on  the  available  prior 
information  (i.e.,  using  the  a  priori  probability  distribution).  However,  the  degree  of 
the  risk  may  be  reduced  by  delaying  such  a  decision  until  additional  information  is 
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obtained.  To  obtain  additional  information,  a  sample  from  the  deviations  is  taken. 

The  value  of  additional  information  obtained  through  sampling  is  equal  to  the 
difference  between  the  expected  economic  loss  of  a  decision  based  only  on  prior 
information  and  the  expected  economic  loss  of  the  same  decision  based  on  the 
additional  information  (i.e.,  a  posteriori  probability  distribution)  plus  the  cost  of 
sampling.  In  short,  if  the  cost  of  the  sampling  is  greater,  the  value  of  obtaining 
additional  information  does  not  justify  the  cost. 

For  instance,  if,  under  a  given  a  priori  probability  distribution,  the  cost  of 
uncertainty  is  very  small,  sampling  is  likely  to  cost  more  than  it  is  wonh.  In  this  case, 
a  decision  can  be  made  without  a  large  degree  of  risk.  Conversely,  if  the  cost  of 
uncertainty  is  large,  the  expected  value  of  additional  information  is  likely  to  be  equal 
to  or  greater  than  the  cost  of  sampling. 

Accounting  control  based  on  modem  decision  theory  takes  advantage  of  the 
expected  value  of  additional  information  obtained  from  investigation.  The  value  of 
this  information  is  derived  by  comparing  the  reduction  of  expected  cost  of  the 
proposed  initial  decision  with  the  cost  of  sampling  and  not  by  the  reduction  of  the 
magnitude  of  the  standard  deviation,  as  in  classical  statistics  (Onsi,  1967:326). 

The  major  advantage  of  this  model  is  that  it  incorporates  an  analytical  tool  to 
reduce  the  degree  of  risk.  However,  several  weaknesses  do  exist.  First,  it  is  difficult 
to  determine  the  a  priori  probability  distribution.  Second,  because  managers’  attitude 
toward  risk  vary,  it  may  be  difficult  to  place  a  monetary  value  on  the  usefulness  of  the 
decision.  This  is  a  problem  when  potentially  large  gains  or  losses  are  expected. 
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To  the  extent  that  the  probability  of  an  event  under  uncertain  conditions  can  be 
determined,  modem  decision  theory  is  a  practical  tool  for  managing  DoD  acquisition 
programs.  Through  management  experience,  subjective  (i.e.,  personal  judgment  based 
on  experience)  probability  distributions  approximating  reality  can  be  developed.  This 
then  provides  a  good  method  for  choosing  the  best  management  alternative  under 
uncertainty  (Onsi,  1%7:325). 

This  chapter  has  examined  three  different  accounting  control  models  and 
accessed  their  applicability  to  DoD  acquisition  programs.  The  goal  of  these  three 
models  is  to  decide  whether  or  not  to  investigate  variations.  The  traditional 
accounting  model  relies  strictly  on  management  judgment  and  experience  to  determine 
an  absolute  value  for  variation  control  limits.  The  second  model  based  on  classical 
statistics  uses  a  mathematically  derived  frequency  distribution  as  a  basis  for  the 
establishment  of  control  limits.  Finally,  a  model  based  on  modem  decision  theory 
uses  subjective  or  personal  probability  for  making  informed  decisions  to  investigate 
variances. 

Statistical  Process  Control  Approach 

Statistical  Process  Control  (SPC)  uses  control  charts  to  monitor  the  outcome  of 
a  process.  These  control  charts  assists  managers  to  identify  and  eliminate  special 
causes  of  variations.  Control  charts  are  derived  through  statistical  process  control 
which  is  a  proven  technique  for  reducing  variations,  thus,  increasing  management 
control  over  the  process  (Evans,  1993:530). 
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Control  charts  were  first  proposed  by  W.A.  Shewhart  in  the  1920s,  who 
belonged  to  the  Bell  Telephone  Laboratories.  The  purpose  of  these  charts  is  to 
eliminate  abnormal  variation  by  distinguishing  variations  due  to  assignable  causes 
from  those  due  to  chance  causes  (Kume,  1985:91). 

A  control  chart  consists  of  a  central  line,  a  pair  of  control  limits  allocated  one 
above  and  one  below  the  central  line.  Characteristic  values  (e.g.  cost  variations)  are 
plotted  on  the  chart  to  represent  the  state  of  a  process.  If  all  these  values  are  plotted 
within  the  control  limits  without  any  particular  tendency,  the  process  is  regarded  as 
being  in  the  controlled  state.  However,  if  they  fall  outside  the  control  limits  or  exhibit 
a  peculiar  form,  the  process  is  considered  to  be  out  of  control  (Figure  5). 

Similar  to  accounting  control  approaches,  the  purpose  of  using  control  chans  is 
to  determine  when  to  take  action  to  adjust  a  process  which  has  fallen  out  of  control. 
Just  as  imponant,  it  also  indicates  when  to  leave  a  process  alone.  This  section 
examines  several  commonly  used  control  chans: 

1)  X-Bar  Chan,  R  Chan,  and  S  Chart 

2)  Cumulative  Sum  (Cusum)  Chart 

3)  The  Economic  X-Bar  Chart  and  Economic  Cusum  Chart 

X-Bar  Chart,  R  Chart,  and  S  Chart.  The  X-Bar  and  the  R  charts  (range  chart) 
are  the  most  commonly  used  control  charts.  These  charts  are  used  together  for  the 
analysis  data  measured  along  a  continuous  scale.  The  X-Bar  Chart  is  used  to  monitor 
the  central  tendency  of  a  process  whereas  the  R  chart  is  used  to  monitor  the  variation 
of  the  process. 
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Figure  5.  A  Typical  Control  Chart 
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In  order  to  construct  these  charts,  samples  of  some  given  sample  size  are  taken 
from  the  process  of  interest.  For  each  sample,  the  average  and  the  range  are 
computed.  Next,  the  average  of  the  averages  (i.e.  overall  process  mean)  and  the 
average  of  the  ranges  (R-Bar)  are  computed.  These  averages  form  the  basis  for  the 
central  lines  of  the  respective  chans.  From  these  central  values  control  limits  are 
computed. 

The  control  limits  are  usually  set  at  three  standard  deviations  from  the  average 
measure.  The  underlining  assumption  for  the  computation  of  the  control  limits  is  that 
the  distribution  of  the  sample  averages  is  normally  distributed  (Evans,  1993:541). 
Therefore,  it  is  expected  that  approximately  99.7  percent  of  the  sample  data  will  fall 
within  these  control  limits  (Figure  6).  If  any  point  fall  outside  the  control  limits  or  if 
any  unusual  patterns  are  observed,  it  is  likely  that  some  assignable  cause  is  affecting 
the  process,  and  an  analysis  should  be  performed  to  determine  that  cause  (Moore  and 
Hendrick,  1992:A.3.1;  Wheeler,  1986:204;  Devore,  1991:415;  Homgren  and  Foster, 
1992:845). 

If  the  assumption  of  a  normal  distribution  for  the  sample  data  holds,  the 
following  rules  can  be  applied  for  examining  a  process  to  determine  if  it  is  in  control: 

1)  No  points  are  outside  the  control  limits. 

2)  The  number  of  points  above  and  below  the  center  line  is  approximately  the 

same. 

3)  The  points  seem  to  fall  randomly  above  and  below  the  center  line. 

4)  Most  points,  but  not  all,  are  near  the  center  line,  and  only  a  few  are  close  to 

the  control  limits  (Evans,  1993:539). 
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Figure  6.  Normal  Distribution  with  3a  Control  Limits 
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An  alternative  to  using  the  R  Chart  along  with  the  X-Bar  Chart  is  to  compute 
and  plot  the  standard  deviation,  i’,  of  each  sample.  Although  the  range  has 
traditionally  been  used,  since  it  involves  less  compui  itional  effort  and  is  easier  to 
understand,  there  is  an  advantage  to  using  s  instead  of  R.  The  sample  standard 
deviation  is  a  more  sensitive  and  better  indicator  of  process  variability  (Evans, 
1993:585).  Therefore,  when  tight  control  of  variability  is  required,  s  should  be  used. 
The  procedure  for  constructing  S  Chart  is  similar  to  the  procedure  for  constructing  the 
R  Chart. 

A  defect  of  the  traditional  X-Bar  Chart  is  its  inability  to  detect  a  relatively 
small  change  in  a  overall  process  mean  (Devore,  1991:653).  This  is  largely  a 
consequence  whether  a  process  is  judged  out  of  control  at  a  particular  time  depends 
only  on  the  sample  at  that  time,  and  not  on  the  past  history  of  the  process,  i  o 
overcome  this  shortcoming,  the  Cumularive  Sum  control  chart  was  designed  to  give 
early  indication  of  process  changes. 

Cumulative  Sum  Chart  (Cusum).  The  Cusum  Chan  incorporates  all  past  data 
by  plotting  cumulative  sums  of  the  deviation  of  sample  values  from  a  target  value. 

The  Cusum  Chan  looks  very  different  from  the  previous  control  charts.  In  place  of  a 
center  line  and  horizontal  control  limits,  a  "V  mask"  is  constructed.  Any  particular  V- 
mask  is  determined  by  specifying  design  parameters  such  as,  the  "lead  distance,"  d, 
and  "half-angle,"  0,  as  illustrated  in  Figure  7.  One  method  for  deciding  which  mask  to 
use  involves  specifying  the  size  of  a  shift  in  the  process  mean  that  is  of  particular 
concern  to  an  investigator.  Then  the  parameters  of  the  mask  are  chosen  to  give 
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desired  values  of  Type  I  and  Type  II  error,  the  false  alarm  probability  and  probability 
of  not  detecting  the  specified  shift,  respectively. 

The  mask  is  located  on  the  chart  so  that  the  point  p  lies  on  the  last  point 
plotted.  If  no  previous  points  lie  outside  the  control  limits,  the  process  is  assumed  to 
be  in  control  (Figure  8-a).  If  there  is  a  shift  in  the  process  mean  above  the  reference 
value,  each  new  value  if  added  to  the  cumulative  sum  will  cause  the  cumulative  sum 
to  increase  and  result  in  an  upward  trend  in  the  chart.  Eventually  a  point  will  fall 
outside  the  upper  control  limit,  indicating  that  the  process  has  fallen  out  of  control 
(Figure  8-b). 

A  major  advantage  of  the  cusum  control  chan  is  the  ease  of  detecting  the  shift 
in  process  mean  through  visual  inspection  of  the  chan.  However,  the  disadvantage  is 
that  is  very  difficult  to  define  appropriate  design  parameters. 

X-Bar  and  Cusum  charts  are  useful  in  determining  if  a  process  is  in  or  out  of 
control  and  any  shift  of  process  mean.  However,  these  chans  ignore  the  cost  and  risk 
associated  with  the  process.  Therefore,  the  Economic  X-Bar  and  Economic  Cusum 
chans  were  developed  to  account  for  these  costs  and  risks. 

Economic  X-Bar  Chart  and  Economic  Cusum  Chart.  Duncan  established  a 
criterion  that  measures  approximately  the  average  net  income  of  a  process  under 
surveillance  of  an  X-Bar  Chan  when  the  process  is  subject  to  random  shifts  in  the 
process  mean.  In  his  analysis,  it  is  assumed  an  assignable  cause  is  investigated 
whenever  a  point  falls  outside  the  control  limits.  The  criterion  is  for  the  case  in  which 
it  is  assumed  that  the  process  is  not  shut  down  while  the  search  for  the  assignable 
cause  is  in  progress,  nor  is  the  cost  of  adjustment  or  repair  and  the  cost  of  bringing 
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Figure  8-b.  Vmask  with  Points  Outside 
Control  Limits 

(Adapted  from  Evans,  1 993:602) 
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the  process  back  into  a  state  of  control  after  the  assignable  cause  is  discovered  charged 
to  the  control  chart  program. 

This  design  maximizes  the  long  run  average  net  income  for  the  process.  It 
assumes  that  the  management  has  knowledge  of  the  risk  of  occurrence  of  an 
assignable  cause,  knowledge  of  various  costs  and  income  parameters.  The  maximum 
income  criterion  is  the  b  is  for  deciding  whether  of  not  to  investigate  variances. 

An  assumption  is  made,  the  control  chart  is  maintained  to  detect  a  single 
assignable  cause  that  occurs  at  random  and  results  in  a  change  in  the  process  of 
known  proportions. 

Even  though  control  charts  as  proposed  by  Shewhart  have  been  in  use  for  over 
sixty  years,  the  increasing  complexity  and  cost  of  industrial  processes  have 
necessitated  a  search  for  more  efficient  and  economical  means  of  improving  quality 
(Goel  and  Wu,  1973:1272).  An  important  development  in  this  direction  was  the 
introduction  of  the  Cusum  procedures  in  1954,  which  are  based  on  sums  of 
observations  rather  than  individual  observations. 

Goel  and  Wu  provided  a  methodology  for  the  economic  design  of  cusum  chans 
to  control  the  mean  of  a  process  with  a  normally  distributed  quality  characteristic 
(Goel  and  Wu,  1973:1281)  .  Unlike  the  previous  cusum  model,  this  model  is  based  on 
minimum  cost  criterion  for  investigating  variances.  A  model  is  derived  which  gives 
the  long-run  average  cost  as  a  function  of  both  the  design  parameters  of  the  chart  and 
the  cost  and  risk  factors  associated  with  the  process. 

These  statistical  process  control  techniques  are  widely  used  in  private  industry 
to  detect  shifts  in  process  mean  and  changes  in  variation  (Evans,  1993:565).  In  a 


similar  way,  control  charts  can  be  used  by  DoD  program  managers  to  monitor  the 
outcome  of  program  cost  and  schedule  management.  Control  charts  help  managers 
assess  the  program  stability  in  order  to  determine  whether  or  not  to  conduct  cost  and 
schedule  variance  analysis. 

Dyckman  Model.  The  Dyckman  model  is  a  two-state  (in  control,  out  of 
control),  two-action  (investigate,  do  not  investigate)  model  using  a  critical  value 
(probability)  approach  to  determine  the  action  required  (Jacobs,  1978:191).  The  model 
employs  a  stochastic  process  and  a  Bayesian  based  decision  making  process  to 
describe  transitions  between  an  in-control  state  and  an  out-of-control  state  and  to 
update  the  probability  of  being  in  either  state  after  each  observation  from  the  process 
(Kaplan,  1975:328). 

Dyckman  assumes  a  constant  savings  from  investigating  an  out-of-control 
process.  He  calls  it  the  "present  value  of  the  savings  obtainable  from  an  investigation 
when  the  activity  is  out-of-control."  This  method  also  considers  the  cost  of  correcting 
an  out-of-control  process  and  the  present  value  of  the  losses  from  not  investigating  an 
out-of-control  process.  The  variance  thresholds  are  determined  by  a  ratio  of  these 
factors. 

However,  the  difficulty  arises  because  Dyckman  "suppresses  the  sequential 
decision-making  nature  of  the  problem"  (Kaplan,  1975:328).  As  a  result,  the  benefit 
from  delaying  the  investigation  for  another  period  when  more  sample  evidence  maybe 
obtained  is  not  evaluated. 

The  application  of  this  model  to  DoD  major  program  management  is  feasible. 
DoD  program  managers  are  responsible  for  the  control  of  the  level  of  several  process 
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variables,  such  as  cost,  schedule,  and  performance.  These  process  variables  may  move 
only  from  a  desirable  state  to  a  less  desirable  state,  e.g.,  schedule  slippage,  cost 
overruns.  Dyckman’s  model  can  be  used  as  a  control  tool  to  indicate  when 
intervention  should  take  place. 

Kaplan  Model.  One  method  developed  by  Kaplan  is  an  extension  of 
Dyckman’s  model.  This  model  is  a  multi-period  model  which  uses  the  actual  costs 
when  operating  in  or  out-of-control  to  establish  optimal  thresholds.  Therefore,  a 
decision  to  delay  investigating  for  one  period  incurs  the  risk  of  operating  an  additional 
period  out-of-control.  That  is,  obtaining  a  cost  realization  from  an  out-of-control 
distribution  rather  than  from  an  in-control  distribution  (Kaplan,  1975:324). 

Kaplan  assumes  the  relevant  information  from  the  prior  observations,  since  the 
last  investigation  was  made,  can  be  summarized  by  a  single  state  variable-the 
probability  that  the  system  is  currently  operating  in-control.  This  variable  is  updated 
after  each  observation  via  Bayes’  theorem  to  incorporate  information  from  the  most 
recent  observation. 

Another  assumption  of  the  model  is  the  simplification  of  the  process  to  a  two- 
state  system,  in-control  and  out-of-control,  with  sudden  transitions  between  the  states. 
Additionally,  Kaplan  assumes  that  an  out-of-state  process  can  always  be  returned  to 
the  in-control  state. 

It  is  these  assumptions  which  may  limit  the  applicability  of  this  model.  First,  a 
forced  dichotomy  between  in-control  and  out-of-control  may  not  be  an  accurate 
description  of  a  reality.  Also,  fundamental  shifts  in  the  process  may  occur  that  lu-e 
not  reversible  even  after  discovery.  For  example,  prices  may  have  risen  which  may  be 
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impossible  to  reverse.  This  feature  represertts  one  of  the  fundamental  differences 
between  the  traditional  quality  control  settings  for  which  most  of  the  models  and  the 
cost  variance  setting  have  been  developed  (Kaplan,  1975:327).  Physical  processes 
monitored  in  the  quality  control  environment  can  almost  be  returned  from  the  out-of¬ 
control  state  to  the  desired  setting  once  such  a  state  is  discovered.  Therefore,  the 
benefits  from  investigating  these  processes  can  be  measured.  Finally,  the  situation 
where  the  investigation  fails  to  detect  and  out-of-control  situation  when  one  exists  is 
not  considered. 

In  general,  this  multi-period  economic  model  is  consistently  more  effective 
than  previous  models  reviewed  although  each  may  have  specific  applications  (Jacobs, 
1978:202).  This  model  might  be  similarly  applied  in  DoD  program  acquisition 
management.  However,  its  assumpiions  might  limit  applicability  to  DoD  scenarios. 
For  instance,  in  the  cost  and  schedule  variance  setting,  the  benefits  from  investigation 
may  be  difficult  to  determine. 

This  literature  review  answered  the  first  two  investigative  questions  addressed 
in  the  introduction  chapter. 

Investigative  Question  #1:  What  models  are  available  for  establishing 
variance  thresholds? 

Based  on  the  conduct  of  a  thorough  literature  review,  there  are  at  least  nine 
models  which  may  be  used  to  establish  cost  and  schedule  variance  thresholds.  The 
following  list  of  models  summarizes  those  examined  in  the  literature  review: 

Traditional  Accounting  Model 

Accounting  Model  Based  on  Classical  Statistical  Theory 
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Accounting  Control  Model  Based  on  Modem  Decision  Theory 

X-Bar  Chart,  R  Chart,  and  S  Chart 

Cumulative  Sum  Chart 

Economic  X-Bar  Chart 

Economic  Cusum  Chart 

Dyckman  Model 

Kaplan  Model 

The  Traditional  Accounting  Model  relies  on  simple  and  informal  ways,  such  as 
management  judgment  and  experience  for  determining  an  absolute  value  variation. 

The  second  model  is  based  on  classical  statistical  theory  rather  than  management 
judgment  and  experience  to  set  cost  and  schedule  variance  thresholds.  This  model 
uses  a  mathematically  derived  frequency  distribution  as  a  basis  for  setting  the  control 
limits.  In  addition,  it  makes  a  distinction  between  chance  variation  and  assignable 
variation,  and  that  only  assignable  should  be  investigated.  The  third  model  based  on 
modem  decision  theory  uses  subjective  probability  for  making  an  informed  decision 
whether  or  not  to  investigate  a  variance. 

The  statistical  process  control  approach  uses  control  charts  to  monitor  the 
outcome  of  a  process.  Its  purpose  is  to  determine  when  to  take  action  to  adju''*  a 
process  which  has  fallen  out  of  control.  This  approach  employs  several  commonly 
used  control  charts,  such  as  X-bar  Chart,  R  Chart,  S  Chart,  Cusum  Chart,  Economic  S- 
Bar  Chart,  and  Economic  Cusum  Chart. 

The  Dyckman  and  Kaplan  models  are  extensions  of  the  control  chart  models. 
The  Dyckman  model  provides  a  means  for  making  the  cost  deviation  investigation 
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decision  that  incorporates  both  the  cost  of  investigation  and  the  expected  savings  from 
returning  the  process  to  an  in-control  state.  It  uses  a  Bayesian  decision  theory 
approach  to  make  that  decision.  The  Kaplan  model  illustrates  how  the  probability 
distribution  of  operating  out  of  control  and  the  various  costs,  the  cost  of  operating  out 
of  control  and  cost  of  investigation,  can  be  integrated  to  yield  the  best  economic 
decision. 

As  evident  from  this  research,  there  are  a  number  of  models  which  can  be  used 
to  help  managers  to  establish  appropriate  cost  and  schedule  variance  thresholds.  From 
these  thresholds,  decisions  can  be  made  to  determine  whether  an  investigation  is 
necessary  to  find  a  cause  for  an  out-of-control  process. 

Investigative  Question  #2:  Are  any  of  these  models  applicable  to  DoD 
major  program  contracts? 

To  some  extent  these  models  can  be  applied  to  the  establishment  of  cost  and 
schedule  variance  thresholds  on  DoD  major  program  contracts.  In  the  authors’ 
judgment,  applicability  of  the  models  can  be  characterized  in  three  broad  categories: 
easy,  moderate,  and  difficult.  Table  3  presents  the  characteristics  of  the  models 
examined  in  Chapter  II  and  the  assessment  of  the  applicability  of  these  models  to  DoD 
major  program  contracts.  This  assessment  of  the  applicability  was  the  result  of 
considering  the  four  characteristics  listed  in  the  table.  These  characteristics  are 
Complexity,  Factors  Considered,  Period,  and  Decision  Approach. 

Complexity  refers  to  the  degree  of  difficulty  in  modeling  the  process  and 
executing  the  model  (e.g.,  mathematical  tools  used,  assumptions  made,  etc.).  The 
authors  assumed  the  more  complex  the  model  the  more  difficult  in  applying  it  to  DoD 
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TABLE  3 


SUMMARY  CHARACTERISTICS 


Model 

i 

2 

3 

4 

App 

Traditional 

Accounting  Model 

Simple 

None 

Single 

M.J. 

Easy 

Accounting  Model 

Mod 

PK, 

Multi 

CS 

Mod 

Based  on  Classical 

Statistical  Theory 

Risk 

Accounting  Control 

Complex 

PK. 

Multi 

BDT 

Diff 

Model  Based  on  Modem 

Cost 

Decision  Theory 

Risk 

X-Bar  Chart, 

R-Chart,  and 

S-Chart 

Simple 

None 

Single 

SPC 

Easy 

Cumulative  Sum  Chart 

Mod 

PK 

MulU 

SPC 

Mod 

Economic 

X-Bar  Chart 

Mod 

Cost 

Single 

SPC 

Mod 

Economic 

Mod 

PK. 

Multi 

SPC 

Mod 

Cusum  Chart 

Cost 

Dyckman  Model 

Complex 

Cost 

Multi 

BDT 

Dili 

Kaplan  Model 

Complex 

PK. 

Multi 

BDT 

Diff 

Cost. 

Risk 

1- Complexity  -  Simple,  Moderate  (Mod),  Complex 

2- Factors  Considered  -  Prior  Knowledge  (PK),  Cost,  Risk 

3- Period  -  Single,  Multiple  (Multi) 

4- Decision  Approach  -  Management  Judgment  (MJ),  Classical  Statistics  (CS), 
Bayesian  Decision  Theory  (BDT),  Statistical  Process  Control  (SPC) 
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major  programs.  The  Factors  considered  included  the  prior  knowledge  required  to 
initiate  the  model,  the  cost,  and  the  risk  of  investigating  a  variance  or  not  investigating 
a  variance.  Period  means  that  the  models  used  data  from  either  a  single  period  or 
more  than  one  period.  The  Decision  Approach  signifies  the  method  by  which  a 
decision  to  investigate  a  variance  was  derived.  These  methods  consist  of  management 
judgment,  classical  statistical  approach,  statistical  process  control  approach,  and 
Bayesian  decision  theory. 

The  Traditional  Accounting  Model  used  to  establish  cost  and  schedule  variance 
thresholds  can  be  easily  applied  to  DoD  major  program  contracts.  The  major 
advantage  of  this  model  was  its  simplicity  because  no  extensive  or  time-consuming 
computations  were  necessary.  Only  managerial  judgment  and  experience  were 
required.  However,  this  model  might  not  prove  as  effective  because  it  did  not 
explicitly  consider  other  factors,  such  as  prior  knowledge,  cost,  or  risk.  Also,  since 
the  model  considered  only  a  single  period,  the  variance  thresholds  were  not  updated  to 
reflect  the  latest  condition  of  the  process.  Though  management  judgment  and 
experience  are  beneficial  for  program  control,  often  in  the  DoD  environment, 
managers  do  not  remain  with  the  same  program  throughout  its  life.  Further, 
experience  and  judgment  vary  among  managers.  This  may  cause  the  level  of  program 
control  to  be  less  effective. 

The  Statistical  Process  Control  models  examined  in  Chapter  11  consisted  of  the 
X-Bar  Chart,  R  Chart,  S  Chart,  Cumulative  Sum  Chart,  Economic  X-Bar  Chart,  and 
Economic  Cusum  Chart.  The  degree  of  difficulty  in  modeling  the  process  for  these 
models  ranged  from  simple  to  moderate  in  complexity.  In  the  less  complex  X-Bar,  R 
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Bar,  and  S  Bar  charts,  decisions  were  made  based  on  a  single  observation.  The 
Economic  X-Bar  Chart  also  used  a  single  observation  but  did  consider  various 
investigation  costs  and  income  data.  In  the  more  complex  Cusum  and  Economic 
Cusum  charts,  decisions  were  based  on  multiple  observations.  Again,  the  Cusum 
Chart  did  not  consider  the  investigation  costs  while  the  Economic  Cusum  Chart  did 
consider  these  costs. 

These  models  are  widely  used  in  pnvate  industry  to  detect  shifts  in  process 
mean  and  changes  in  variation  (Evans,  1993:530).  With  some  basic  knowledge  in 
statistics,  the  decision  maker  can  easily  apply  the  X-Bar  Chart,  R  Chart,  and  S  Chart 
to  DoD  major  programs.  However,  while  this  model  may  be  simple  to  use,  it  is 
similar  to  the  Traditional  Accounting  Model  in  that  it  ignored  prior  knowledge,  cost, 
and  risk. 

The  other  three  SPC  models.  Economic  X-Bar  Chart,  Cusum  Chart,  and 
Economic  Cusum  Chart,  are  relatively  more  difficult  to  apply  to  DoD  major  programs. 
This  is  due  mainly  to  a  higher  level  of  complexity  required  to  develop  the  models. 

For  example,  the  models  required  additional  information  such  as,  prior  knowledge, 
cost  data,  and  an  estimation  of  the  risk  of  investigating  a  variance. 

In  general,  the  authors  believe  control  charts  can  be  used  by  DoD  program 
managers  to  monitor  the  outcome  of  program  cost  and  schedule  management.  Control 
charts  help  managers  assess  the  program  stability  in  order  to  determine  whether  or  not 
to  conduct  cost  and  schedule  variance  analysis. 

The  Accounting  Model  based  on  Classical  Statistical  Theory  is  also  moderately 
applicable  to  DoD  major  programs.  Similar  to  Cusum  models,  this  model  utilized 
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multiple  observation  data  and  considered  all  risks  associated  with  the  decision  to 
investigate  a  variance.  Again,  a  basic  knowledge  of  statistics  was  required  to  develop 
and  execute  the  model.  Moreover,  since  this  model  considered  multiple  periods,  it  is 
often  difficult  and  time-consuming  to  collect  the  necessary  data. 

The  other  three  models  which  used  a  Bayesian  decision  approach  are 
potentially  difficult  to  apply  to  DoD  programs  due  to  their  complexity.  First,  these 
models  required  an  extensive  know'ledge  in  statistics  to  develop  and  execute.  Second, 
the  development  and  implementation  costs  were  potentially  higher  because  of  the 
additional  expertise  required  and  the  time  required  to  gather  data  and  develop  these 
models. 

In  short,  the  degree  of  applicability  of  threshold  models  to  DoD  programs  is 
mainly  dependent  upon  the  complexity  of  each  model  and  the  time  and  costs 
associated  with  the  development  and  implementation  of  these  models.  It  seems  that, 
there  is  no  single  best  model  for  all  programs. 

Summary 

In  the  accounting  approach,  the  traditional  accounting  model  relies  on  simple 
and  informal  ways,  such  as  management  judgment  and  experience  for  determining  a 
absolute  value  variation.  Another  model  based  on  statistical  theory  rather  than 
management  judgment  and  experience  to  set  cost  and  schedule  variance  thresholds. 
This  model  uses  a  mathematically  derived  frequency  distribution  as  a  basis  for  setting 
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the  control  limits.  In  addition,  it  makes  a  distinction  between  chance  variation  and 


assignable  variation,  and  that  only  assignable  should  be  investigated. 

The  statistical  process  control  approach  uses  control  charts  to  monitor  the 
outcome  cf  a  process  and  to  determine  when  to  take  action  to  adjust  a  process  which 
has  fallen  out  of  control.  This  approach  employs  several  commonly  used  control 
charts,  such  as  X-Bar  Chart,  R  Chart,  S  Chart,  Cusum  Chart,  Economic  X-Bar  Chart, 
and  Economic  Cusum  Chan,  for  its  purpose. 

The  Dyckman  and  Kapian  models  are  extensions  of  the  control  chart  models. 
Dyckman  model  provides  a  means  for  making  the  cost  deviation  investigation  decision 
that  incorporate  both  the  cost  of  investigation  and  the  expected  savings.  It  uses  a 
statistical  decision  theory  approach  to  make  that  decision.  Kaplan  model  illustrates 
how  the  probability  distribution  of  operating  out  of  control  and  the  vanous  costs:  the 
cost  of  operating  out  of  control  and  cost  of  investigation  can  be  integrated  to  yield  the 
best  economic  decision. 

These  models  can  be  used  to  established  cost  and  schedule  variance  thresholds 
in  DoD  major  programs.  In  authors’  judgment,  the  level  of  applicability  of  these 
models  depends  largely  on  the  complexity  of  each  model  and  the  time  and  cost 
associated  with  the  development  and  implementation  of  these  models. 

The  next  chapter  will  describe  the  specific  procedures  for  conducting  the 
research.  The  chapter  will  cover  the  following  subtopics: 

1)  Research  Findings 

2)  Population 

3)  Sample 
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4)  Instrument  Development 

5)  Data  Collection. 
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HI.  Methodology 


Introduction 

This  chapter  explains  the  process  used  to  collect  the  information  necessary  to 
answer  the  investigative  and  research  questions  discussed  in  Chapter  I. 

Research  Design 

Different  interpretations  of  variance  thresholds  between  government  and 
contractor  personnel  may  present  problems  controlling  contract  cost  and  schedule. 
These  different  interpretations  indicate  the  need  for  setting  appropriate  variance 
thresholds.  Recognizing  this  fact,  researchers  have  developed  many  theoretical  models 
as  described  in  the  previous  chapter  to  help  establish  such  thresholds.  Although  cost 
and  schedule  variance  thresholds  are  required  on  DoD  major  program  contracts,  little 
is  known  about  the  methods  the  government  and  contractors  use  to  establish  these 
threshold  values. 

The  purpose  of  this  thesis  is: 

1)  to  examine  to  what  extent  theoretical  models  are  used  to  establish  cost  and 
schedule  variance  thresholds  on  DoD  major  program  contracts, 

2)  to  identify  how  cost  and  schedule  variance  thresholds  are  established  on 
DoD  major  program  contracts,  and 

3)  to  report  those  findings. 
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This  thesis  is  descriptive  in  nature.  It  examines  things  as  they  exist  and  does  not 
attempt  to  manipulate  any  variables. 

This  thesis  is  a  formalized  study  consisting  of  eight  structured  investigative 
questions.  These  investigative  questions  are: 

1)  What  models  are  available  for  establishing  variance  thresholds? 

2)  Are  any  of  these  models  applicable  to  DoD  major  program  contracts? 

3)  Who  is  responsible  for  setting  cost  and  schedule  variance  thresholds  on  DoD 
major  program  contracts? 

4)  Are  DoD  personnel  aware  of  these  models? 

5)  Are  any  of  these  models  being  used  by  the  DoD? 

6)  How  are  cost  and  schedule  variance  thresholds  established  on  DoD  major 
program  contracts? 

7)  What  models  for  establishing  cost  and  schedule  variance  thresholds  are 

used? 

8)  Do  the  chosen  methods  vary  depending  on  program  type,  preparation  time, 
risk,  contract  phase,  and  contract  cost? 

A  thorough  literature  review  on  the  topic  of  variance  thresholds  in  the  last 
chapter  answered  the  first  two  investigative  questions.  The  chapter  also  discussed  the 
advantages,  disadvantages,  differences,  and  the  applicability  of  these  models  to  DoD 
major  program  contracts.  The  remainder  of  the  investigative  questions  required 
personal  interviews  with  government  and  contractor  personnel. 
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Population 


Based  on  the  research  objective,  the  relevant  population  of  interest  was 
determined.  This  population  included  all  government  financial  managers  and  civilian 
contractor  counterparts  who  had  first-hand  knowledge  of  the  establishment  of  cost  and 
schedule  variance  thresholds  for  DoD  major  program  contracts. 

According  to  a  memorandum  from  the  Office  of  the  Under  Secretary  of 
Defense  for  Acquisition  addressed  to  the  service  secretaries  dated  9  February  1990, 
there  were  123  U.S.  Army,  USAF,  U.S.  Navy,  and  DoD  major  programs  (Appendix 
A).  Based  on  the  authors’  observations,  an  average  of  two  financial  managers  and  two 
contractor  counterparts  for  each  DoD  major  program  was  a  reasonable  estimate. 
Therefore,  the  total  number  of  personnel  involved  in  the  establishment  of  cost  and 
schedule  variance  thresholds  for  these  programs  was  estimated  to  be  approximately 
492. 

Since  the  release  of  the  above  memorandum,  a  number  of  major  programs  have 
been  canceled  (e.g.,  A- 12,  Rail  Garrison,  Small  Missile)  due  various  reasons,  such  as 
poor  management  and  the  reduced  threat  from  the  communist  block.  These 
cancellations  coupled  with  DoD  and  defense  industry  personnel  reductions  rendered 
this  population  estimate  highly  debatable. 

Sample 


For  the  scope  of  the  thesis,  elements  of  this  sample  were  DoD  system  program 
office  personnel.  Additionally,  .several  defen.se  contractor  personnel  were  interviewed. 
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This  sample  consisted  of  USAF  Aeronautical  Systems  Center  (ASC)  Financial 
Management  Staff  personnel,  USAF  System  Program  Office  (SPO)  personnel  at 
Wright-Patterson  AFB,  U.S.  Army  and  U.S.  Navy  program  office  personnel,  and 
defense  contractor  personnel.  Even  though  the  population  size  estimate  was 
subjective,  it  was  the  goal  of  the  authors  to  include  as  many  samples  as  possible  given 
the  authors’  time  and  funding  constraints.  As  this  thesis  was  descriptive  in  nature, 
rigorous  statistical  methods  to  manipulate  data  were  not  used.  This  limited  the 
inferences  which  could  be  drawn  concerning  the  population. 

A  judgmental  decision  on  the  sample  was  made  due  to  several  factors: 

1)  Almost  all  USAF  major  aeronautical  program  contracting  is  done  at  Wright- 
Patterson  AFB. 

2)  Data  are  readily  available  for  collection. 

3)  There  is  a  large  government  and  contractor  personnel  pool  for  interview. 

Instrument  Development 

The  research  question  was  to  discover  to  what  extent  the  theoretical  models 
were  used  to  establish  cost  and  schedule  variance  thresholds  on  DoD  major  program 
contracts,  and  eight  specific  investigative  questions  were  used  to  answer  this  research 
question.  In  addition,  four  measurement  questions  were  derived  from  the  investigative 
questions  and  were  posed  to  each  interviewee.  The  first  of  these  questions  was 
dichotomous  to  determined  if  the  interviewee  possessed  first-hand  knowledge  of  the 
establishment  of  variance  thresholds.  A  negative  response  to  this  question  would 
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terminate  the  interview  as  the  interviewee  would  not  be  representative  of  the 
population  of  interest.  The  remainder  of  the  measurement  questions  were  open-ended 
to  facilitate  discussion.  For  example,  the  fourth  measurement  question  was  designed 
to  include  all  potential  factors,  such  as  contract  costs  and  program  type,  which  might 
affect  the  establishment  of  cost  and  schedule  variance  thresholds. 

These  measurement  questions  were: 

1)  Do  you  establish  cost  and/or  schedule  variance  thresholds  on  any  DoD 
major  program  contracts? 

2)  What  method  or  methods  do  you  use? 

3)  What  other  theoretical  models  for  establishing  variance  thresholds  are  you 
aware  of? 

4)  What  factors  (i.e.,  program  type,  preparation  time,  risk,  contract  phase, 
contract  cost)  do  you  consider  when  setting  cost  or  schedule  variance  thresholds? 

All  the  questions  were  structured  with  the  following  factors  considered  to 
minimize  measurement  errors  and  respondent  confusion  (Emory,  1991:362). 

1)  Shared  vocabulary  -  Technical  terms  such  as  variance  thresholds,  C/SCSC, 
and  variance  analysis  were  understood  by  all  respondents.  This  is  verified  at  the  time 
of  each  interview. 

2)  Clarity  -  The  questions  posed  contained  very  little  ambiguous  wording. 

They  v  ^'e  short,  direct,  and  focused  toward  a  single  element. 

3)  Hidden  assumptions  -  No  hidden  assumptions.  Measurement  questions 
established  the  competence  level  of  interviewees  and  minimized  measurement  error. 
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4)  Biased  wording  -  Bias  was  minimized  by  asking  all  respondents  the  same 
questions.  No  superlatives,  slang  expressions  or  fad  words  were  used. 

5)  Personalization  -  Personalization  was  chosen  to  put  interviewees  at  ease  and 
to  promote  the  flow  of  information.  Questions  were  worded  with  second  personal 
pronouns. 

6)  Adequate  alternatives  -  All  but  the  first  measurement  questions  were  open- 
ended,  again  to  promote  the  flow  of  information. 

Additionally,  the  questions  were  arranged  in  such  a  manner  as  to  encourage  the 
respondents’  interest  in  the  topic.  For  instance,  the  furst  question  was  designed  to 
stimulate  the  respondents’  attention  to  and  interest  in  C/SCSC,  specifically  in  variance 
analysis  and  variance  thresholds.  Moreover,  the  first  question  ensured  the 
interviewee’s  subject  knowledge.  They  were  sequenced  in  a  logical  order  and  range 
from  simple  to  complex,  general  to  specific  (Emory,  1991:371). 

Another  consideration  in  instrument  development  is  whether  the  purpose  of  the 
study  should  be  disguised.  The  accepted  wisdom  is  that  knowledge  of  the  purpose  of 
the  research  may  bias  the  results  (Emory,  1991:352).  However,  if  respondents  are 
aware  of  the  study’s  purpose  and  perceive  that  the  topic  is  relevant  to  their  own 
interest,  they  are  more  willing  to  provide  information  (Emory,  1991:359).  Thus,  in 
this  study,  there  was  no  reason  to  disguise  the  objectives,  and  interviewees  were  made 
aware  of  its  purpose. 
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Data  Collection 


Questioning  was  the  basic  methodology  used  in  this  study.  Tl 
selected  for  questioning  the  study  participants  was  the  personal  interv 
personal  interview  offered  several  advantages  to  this  study.  First,  dat 
available  as  almost  all  USAF  major  program  contracting  is  done  at  W 
AFB.  For  this  reason,  there  was  a  large  government  and  contractor  p 
interviews.  In  addition,  to  gather  information  from  other  DoD  agenci 
interviews  was  used. 

For  personal  interviews,  each  sample  element  was  contacted  b 
arrange  a  convenient  time  for  an  interview.  After  this  initial  contact, 
was  conducted  by  the  authors.  At  the  beginning  of  each  interview,  th 
were  given  a  copy  of  the  interview  agenda.  This  agenda  included  the 
study  and  the  designed  measurement  questions.  Responses  were  reco 
interviewers  and  later  compared  for  consistency  and  accuracy. 

Telephone  interviews  were  conducted  in  much  the  same  manni 
interviews.  Each  sample  element  was  contacted  by  telephone  to  arran 
time  for  an  interview  After  this  initial  contact,  each  interview  was  c< 
of  the  authors.  At  the  beginning  of  each  interview,  the  participants  w 
the  interview  agenda  which  included  the  purpose  of  the  study  and  the 
measurement  questions.  Responses  were  recorded  by  the  interviewer. 
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To  ensure  consisten 
measurement  questions  wa^ 
questions  were  posed  by  th 
Information  gathere 
answers  to  the  eight  investi 
information  have  also  been 
the  presented  material.  Th 
All  participants  in  t 
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Summary  and  Overview 

Four  measurement  t 
These  questions  were  poset 
personnel  who  had  first-har 
variance  thresholds.  Questi 
respondents’  confusion.  Ini 
questions  were  collected.  T 


nd  accurate  results  for  data  collection,  the  same  set  of 
pplied  to  all  elements  of  the  sample.  Moreover,  these 
iame  investigators. 

fom  interviewees  was  studied  and  analyzed  to  provide 
-tive  questions.  Tables  presenting  the  research 
icluded  when  they  add  to  the  reader’s  understanding  of 
indings  and  analysis  are  presented  in  the  next  chapter, 
study  were  given  anonymity  and  no  associations  were 
'IS  and  organization  that  are  involved  in  the  study.  The 
ty  questions  they  felt  uncomfortable  with  answering 
viewee  express  such  a  discomfort;  as  a  result,  no 
ly  interview. 


stions  were  derived  to  address  the  research  objective. 

)  a  sample  of  DoD  government  and  civilian  contractor 
knowledge  of  the  establishment  of  cost  and  schedule 
s  were  structured  to  minimize  measurement  errors  and 
viewee  responses  based  on  the  four  measurement 
•  findings  and  analysis  are  presented  in  the  next  chapter. 
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IV.  Research  Findings  and  Analysis 


Introduction 

This  chapter  documents  the  information  obtained  from  following  the  research 
methodology  as  outlined  in  the  previous  chapter.  Information  was  gathered  from: 

1)  A  comprehensive  literature  review  of  theoretical  models. 

2)  Six  personal  interviews  conducted  with  individuals  from  USAF  AFMC/ASC, 
U.S.  Army,  U.S.  Navy,  and  defense  contractors  who  had  first-hand  knowledge  of  the 
establishment  of  cost  and  schedule  variance  thresholds  (Appendix  B). 

3)  A  review  of  nine  defense  contractor  system  description  documents. 

4)  A  review  of  The  U.S.  Army  study.  Establishment  of  Consistent  Variance 
Thresholds  For  Problem  Analysis  on  the  Cost  Performance  Report  for  the  LoAD  PPD 
Program  Phase  IB. 

5)  A  review  of  DoD  Manual  5(XX).2-M,  Defense  Acquisition  Management 
Documentation  and  Reports. 

6)  A  review  of  the  Cost/Schedule  Control  Systems  Criteria  Joint 
Implementation  Guide. 

In  general,  this  chapter  will  answer  -he  following  investigative  questions: 

1)  What  models  are  available  for  establishing  variance  thresholds? 

2)  Are  any  of  these  models  applicable  to  DoD  major  program  contracts? 

3)  Who  is  responsible  for  .setting  cost  and  schedule  variance  thresholds  on  DoD 
major  program  contracts? 
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4)  Are  DoD  personnel  aware  of  these  models? 

5)  Are  any  of  these  models  being  used  by  the  DoD? 

6)  How  are  cost  and  schedule  variance  thresholds  established  on  DoD  major 
program  contracts? 

7)  What  models  for  establishing  cost  and  schedule  variance  thresholds  are 

used? 

8)  Do  the  chosen  methods  vary  depending  on  program  type,  preparation  time, 
risk,  contract  phase,  and  contract  cost? 

Research  Findings 

The  first  two  investigative  questions  were  answered  in  Chapter  II,  Literature 
Review.  This  section  will  address  the  six  remaining  investigative  questions. 

Investigative  Question  #3:  Who  is  responsible  for  setting  cost  and  schedule 
variance  thresholds  on  DoD  major  program  contracts? 

According  to  interviews  conducted  with  financial  management  personnel  at 
USAF  AFMC/ASC  staff  level,  USAF,  U.S.  Army,  and  U.S.  Navy  SPOs,  cost  and 
schedule  variance  thresholds  on  DoD  major  program  contracts  were  established  by  the 
financial  managers  of  each  SPO. 

While  ASC  staff  personnel  did  not  directly  supervise  SPO  financial  personnel, 
they  did  exert  some  degree  of  functional  authority  over  financial  matters.  For 
instance,  an  ASC  policy  letter  regarding  the  establishment  of  cost  and  schedule 
variance  thresholds  was  issued  to  the  various  SPOs  (Appendix  C). 
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Based  on  inputs  from  various  product  divisions,  the  experience  from  ASC  staff 
cost  analysts,  and  collected  sample  financial  Contract  Data  Requirements  Lists 
(CDRLs),  this  policy  letter  provided  lessons  learned  concerning  preparation  of 
financial  CDRLs  and  suggested  approaches  for  establishing  reporting  levels,  variance 
thresholds,  etc.,  with  explanations  as  to  why  the  approach  was  recommended. 

Investigative  Question  #4:  Are  DoD  personnel  aware  of  these  models? 

Interview  results  indicated  that  none  of  the  interviewees  had  knowledge  of  the 
existence  of  various  models  examined  in  Chapter  II.  In  addition,  interviewees 
indicated  that  they  were  not  aware  of  any  other  type  of  models  which  were  not 
examined  in  this  thesis  and  which  might  be  used  to  establish  cost  and  schedule 
variance  thresholds  on  DoD  major  program  contracts. 

Investigative  Question  #5;  Are  any  of  these  models  being  used  by  the  DoD? 

Basically,  none  of  these  models  was  being  used  by  the  DoD,  Even  though 
interviewees  stated  that  they  were  not  aware  of  or  using  any  of  the  specific  models 
examined,  interview  results  suggested  otherwise.  Interview  results  suggested  that  the 
method  used  by  some  of  these  personnel  was  similar  to  the  Traditional  Accounting 
Model.  For  some  interviewees,  managerial  experience  and  judgment  were  the  tools 
used  to  set  cost  and  schedule  variance  thresholds. 

Investigative  Question  #6:  How  are  cost  and  schedule  variance  thresholds 
established  on  DoD  major  program  contracts? 

Research  into  the  fiscal  year  1983  DoD  Authorization  Act  showed  a  serious 
commitment  to  the  reporting  requirements  of  the  DoD  to  the  United  States  Congress. 
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By  law  the  DoD  must  report  to  the  Congress  any  time  a  program  exceeds  its  baseline 
values  by  pre-established  thresholds. 

This  public  law  known  as  the  "Nunn-McCurdy  Amendment"  requires  a  baseline 
which  reflects  the  estimated  worth  of  the  program  in  the  President’s  budget  (U.S. 
Congress,  1982:1557).  This  baseline  is  expressed  in  two  types  of  "unit  cost" 
estimates.  The  two  types  of  unit  costs  are  defined  as: 

(1)  Program  Acquisition  Unit  Cost  (PAUC)-  Representing  the  sum  of  all 
RDT&E,  production,  and  weapon  system  specific  military  construction 
costs  for  the  total  acquisition  program,  divided  by  the  total  program 
acquisition  quantity;  and/or 

(2)  Current  Procurement  Unit  Cost  (CPUC)-Representing  the  total  of  all 
procurement  funds  appropriated  for  the  program  for  a  given  year, 
divided  by  the  number  of  end  items  to  be  procured  in  that  same  year. 

(Fleming,  1992:216) 

Thus,  a  given  program  will  have  on  record  with  the  DoD  and  the  Congress  a 
baseline  of  "total  program  unit  cost"  (PAUC),  and  a  "yearly  unit  cost"  value  (CPUC), 
both  of  which  will  be  monitored  for  the  life  of  the  program.  Two  threshold  limits  are 
prescribed:  a  15%  and  a  25%  level.  Both  these  levels  require  specific  reporting 
actions  by  the  DoD  when  they  are  exceeded.  When  a  program  manager  reports  that  a 
breach  exceeding  15%  has  occurred,  the  service  secretary  must  submit  a  report  to 
Congress.  If  the  breach  exceeds  25%,  then  the  Secretary  of  Defense  must  take  action 
to  assure  Congress  that  the  program  is  essential  to  national  defense  and  is  the  least 
cost  alternative,  and  that  the  management  team  is  still  capable  of  managing  and 
controlling  the  program  costs  (Fleming,  1992:218). 

The  Nunn-McCurdy  Amendment  invariably  affected  DoD’s  internal 
management  regarding  the  establishment  of  variance  thresholds.  DoD  5(KK).2-M,  part 
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20,  Cost  Management  Reports,  paragraph  7.b.5  stated:  "All  reporting  provisions  will 
be  negotiated  and  specified  in  the  contract,  including  reponing  frequency,  specific 
variance  analysis  thresholds,  and  the  contract  WBS  elements  to  be  reported."  This 
was  the  only  reference  to  variance  thresholds  found  in  this  manual.  No  specific 
guidance  regarding  the  actual  formulation  of  variance  thresholds  was  addressed. 

The  C/SCSC  JIG  paragraph  ae,  page  2-3  stated:  "appropriate  thresholds  should 
be  established  as  to  the  magnitude  of  variances  which  will  require  variance  analysis." 
Paragraph  e,  page  3-17  went  on  to  state:  "it  is  essential  that  these  internal  variance 
thresholds  be  so  established  that  all  significant  variances  will  be  analyzed  while  at  the 
same  time  avoiding  an  excessive  number  of  variance  analyses."  This  is  to  minimize 
the  generation  of  analyses  and  explanations  of  variances  which  do  not  have  potential 
for  negative  impact  on  the  program. 

Furthermore,  the  JIG  mentioned  that:  "no  particular  approach  or  set  of 
thresholds  is  ’best’  for  all  circumstances"  (JIG,  1987:3-17).  Different  thresholds 
should  be  considered  for  different  levels  of  management,  for  different  organizational 
elements,  and  for  reporting  to  the  DoD.  Concerning  the  establishment  of  variance 
thresholds,  the  guide  stated:  "generally,  thresholds  are  established  requiring  a  variance 
analysis  for  any  cost  or  schedule  variance  that  exceeds  a  cenain  percentage  of  the 
budget  of  work  scheduled  or  work  performed  and/or  exceeds  an  established  dollar 
minimum"  or  "to  set  the  thresholds  as  a  percentage  of  the  budget  for  the  entire 
project."  It  is  clear  the  guide  emphasized  the  importance  of  setting  appropriate 
variance  thresholds.  However,  the  guide  provided  no  specific  means  or  technique  to 
establish  cost  and  schedule  variance  thresholds. 
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A  review  of  nine  major  defense  contractor  system  descriptions  was  conducted. 
Contractor  system  descriptions  are  documents  which  detail  to  the  government  all  the 
managerial  and  accounting  procedures  the  contractor  uses  to  manage  internally  during 
the  contract.  Pertinent  excerpts  of  these  documents  are  included  in  Appendix  D. 

Although  the  importance  of  setting  appropriate  variance  thresholds  has  been 
illustrated  in  Chapter  I,  of  the  nine  system  descriptions  reviewed  by  the  authors,  none 
addressed  this  topic  in  any  specific  detail.  For  instance,  one  of  the  documents 
contained  over  300  pages.  Four  separate  pages  described  the  requirement  of  variance 
analysis.  However,  not  a  single  sentence  was  devoted  to  the  establishment  of  variance 
thresholds.  In  another  case,  four  pages  out  of  153  pages  described  several  topics 
related  to  variance  thresholds.  The  issues  described  included  variance  analysis, 
variance  analysis  reporting,  and  variance  computation  (e.g..  Cost  Variance  =  Budgeted 
Cost  of  Work  Performed  -  Actual  Cost  of  Work  Performed).  Again,  no  specific 
techniques  for  establishing  thresholds  were  described. 

When  the  establishment  of  variance  thresholds  was  addressed,  the  instructions 
were  very  general.  For  instance,  one  contractor  suggested  using  a  percentage  and  an 
absolute  dollar  value  as  variance  thresholds  stating  "the  cost  account  threshold  are 
usually  defined  by  both  a  percentage  figure  and  a  resource  value  figure  in  hours  or 
dollars  (e.g.,  +/-  10  percent  and  10(K)  hours)."  However,  there  was  no  indication  as  to 
how  these  values  should  be  derived. 

Another  way  in  which  contractors  addressed  the  establishment  of  variance 
thresholds  is  through  contract  direction.  One  system  description  stated:  "variance 
thresholds  may  be  established  at  the  reporting  level  by  the  customer  via  contract 
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direction."  Another  stated:  "at  the  repon  level,  tolerances  are  contractually  established 
for  each  program."  This  implies  that  cost  and  schedule  variance  thresholds  are 
established  by  government  direction.  Also,  variance  thresholds  were  established  by 
contract  negotiation.  One  interviewee  related  that  variance  thresholds  were  agreed 
upon  and  documented  on  the  back  of  his  business  card.  As  a  matter  of  fact,  in  the 
documents  reviewed,  no  specific  calculation  methods  for  the  establishment  of  cost  and 
schedule  variance  thresholds  were  described. 

In  contrast  to  the  documents  described  above,  a  U.S.  Army  study  titled  The 
Establishment  of  Consistent  Variance  TTiresholds  For  Problem  Analysis  on  the  Cost 
Performance  Report  for  the  LoAD  PPD  Program  Phase  IB  suggested  a  mathematical 
model  to  set  variance  thresholds  (Appendix  E).  According  to  this  model,  variance 
thresholds  should  be  a  function  of  the  amount  of  work  performed  in  relation  to  the 
estimated  cost  at  completion  (i.e.  the  thresholds  should  change  over  time).  In  practice, 
contracts  frequently  call  for  the  same  thresholds  to  be  used  throughout  the  life  of  the 
contract.  A  survey  conducted  by  the  National  Security  Industrial  Association  (NSIA) 
Cost/Schedule  System  Compendium  in  1979  supported  the  Army  study.  The  NSIA 
survey  asserted  using  the  same  threshold  throughout  the  life  of  the  contract  "tends  to 
place  emphasis  on  minor  variances  in  low  value  items  while  ignoring  major  variance 
in  high  value  items"  (NSIA  Management  Systems  Subcommittee,  1980:1-14). 

Additionally,  the  Army  model  compensated  for  cost  variations  between 
different  WBS  levels  in  a  contract.  The  study  stated  that  "these  thresholds  shall  be 
applied  routinely  at  level  three  and  above  the  WBS."  The  application  to  lower-level 
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WBS  elements  shall  be  based  on  specific  government  management  needs  for  limited 
time  periods. 

As  addressed  in  Investigative  Question  #3,  ASC  staff  personnel  issued  a  policy 
letter  to  the  various  SPOs  regarding  lessons  learned  for  preparing  financial  CDRLs. 
With  regard  to  the  establishment  of  cost  and  schedule  vari  mce  thresholds,  this  policy 
letter  suggested  approaches  for  establishing  reporting  levels,  variance  thresholds,  etc., 
with  explanations  as  to  why  the  approach  was  recommended. 

Specifically,  the  policy  recommended  establishing  thresholds  using  both  an 
absolute  dollar  amount  and  a  percentage.  For  example,  personnel  were  to  use  a 
percentage  threshold,  such  as  +/-  10  percent,  or  combined  it  with  a  minimum  dollar 
value.  When  an  element  breaches  both  thresholds,  written  analysis  is  required. 
Furthermore,  the  letter  stated,  "when  creating  thresholds,  remember  that  -15  percent  is 
unsatisfactory  when  reponing  to  higher  headquaners."  Again,  the  letter  failed  to 
instruct  personnel  how  these  threshold  levels  should  be  derived. 

Interview  results  indicate  that  no  models  were  explicitly  used  to  establish  cost 
and  schedule  thresholds  on  DoD  major  program  contracts.  For  some  interviewees, 
managerial  experience  and  judgment  were  the  only  tools  used  to  set  these  thresholds. 

One  interviewee  indicated  that  as  far  as  he  could  remember,  variance  thresholds 
for  new  contracts  were  established  by  using  variance  thresholds  from  previous 
contracts.  Two  other  interviewees  supported  this  view.  Additionally,  this  was  a  very 
subjective  process.  If  the  SPO  personnel  perceived  no  problem  with  the  previous 
threshold  levels,  he  would  continued  to  use  the  same  thresholds  for  other  contracts. 
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A  seasoned  financial  manager  with  over  30  years  of  experience,  used  his 
experience  rather  than  any  formal  model  to  establish  cost  and  schedule  variance 
thresholds.  In  a  similar  manner,  another  financial  manager  conceded  that  the 
establishment  of  variance  thresholds  was  not  viewed  as  a  structured  set  of  values.  In 
his  opinion,  it  was  viewed  as  "a  philosophical  approach  with  a  touch  of  personal 
angle"  and  "requires  no  brain."  Again,  past  experience  was  the  dominant  factor  when 
setting  threshold  levels. 

The  research  results  revealed  that  cost  and  schedule  variance  thresholds  were 
essentially  established  on  DoD  major  program  contracts  by  using  management 
experience.  No  specific  guidance  to  formally  establish  variance  thresholds  was  found. 

Investigative  Question  #7:  What  models  for  establishing  cost  and  schedule 
variance  thresholds  are  used? 

Basically,  none  of  these  models  are  being  used  by  the  DoD.  Interviewees 
stated  that  they  were  not  aware  of  or  using  any  of  the  specific  models  examined. 
Interview  results  suggested  that  the  method  used  by  some  of  these  personnel  was 
similar  to  the  Traditional  Accounting  Model. 

Investigative  Question  #8:  Do  the  chosen  methods  vary  depending  on 
program  type,  preparation  time,  risk,  contract  phase,  and  contract  cost? 

As  evident  through  the  review  of  numerous  government  and  defense  contractor 
documents  and  interviews  with  knowledgeable  government  and  defense  contractor 
personnel,  no  specific  methods  examined  in  this  thesis  were  chosen  to  establish  cost 
and  schedule  variance  thresholds.  Therefore,  these  variables,  program  type, 
preparation  time,  risk,  contract  phase,  and  contract  cost,  did  not  affect  the  choice  of  a 
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method  to  establish  cost  and  schedule  variance  thresholds.  Three  interviewees  related 


that  none  of  these  variables  affected  their  choice  of  method. 

Even  though  interviewees  did  not  use  any  of  the  models  examined  to  establish 
cost  and  schedule  variance  thresholds,  soriK  did  consider  these  variables  when 
selecting  threshold  levels  for  a  new  contract.  For  instance,  according  to  one 
interviewee,  contract  phase  was  a  factor  in  setting  variance  thresholds.  Cost  and 
schedule  variance  thresholds  remain  the  same  throughout  the  contract  life  for  the 
current  month  values.  However,  these  thresholds  may  vary  throughout  the  life  of  the 
contract  for  cumulative  amounts.  This  policy  is  consistent  with  the  guidance  set  forth 
in  the  C/SCSC  JIG,  para  3-5e(2)  which  states:  "When  initially  establishing  the 
thresholds,  it  may  be  advisable  to  provide  for  tightening  these  thresholds  as  the 
contract  progresses.  .  ." 

Summary 

This  chapter  used  the  information  gathered  to  answer  the  following 
investigative  questions: 

1)  What  models  are  available  for  establishing  variance  thresholds? 

2)  Are  any  of  these  models  applicable  to  DoD  major  program  contracts? 

3)  Who  is  responsible  for  setting  cost  and  schedule  variance  thresholds  on  DoD 
major  program  contracts? 

4)  Are  DoD  personnel  aware  of  these  models? 

5)  Are  any  of  the.se  models  being  used  by  the  DoD? 
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6)  How  are  cost  and  schedule  variance  thresholds  established  on  DoD  major 
program  contracts? 

7)  What  models  for  establishing  cost  and  schedule  variance  thresholds  are 

used? 

8)  Do  the  chosen  methods  vary  depending  on  program  type,  preparation  time, 
risk,  contract  phase,  and  contract  cost? 

The  findings  indicated  that  while  there  were  many  methods  for  establishing 
cost  and  schedule  variance  thresholds  which  could  be  applicable  to  DoD  major 
program  contracts,  none  of  these  methods  was  being  used.  Repeatedly,  interviewees 
related  that  threshold  levels  for  new  contracts  were  established  by  either  management 
experience  and  judgment  or  by  copying  threshold  levels  from  previous  contracts. 

The  next  chapter  will  provide  conclusion  and  recommendations. 
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V.  Conclusions  and  Recommendations 


Introduction 

This  chapter  uses  the  findings  based  on  the  research  presented  in  Chapters  II 
and  IV  in  order  to  draw  conclusions  regarding  to  what  extent  the  theoretical  threshold 
models  were  used  to  establish  cost  and  schedule  variance  thresholds  on  DoD  major 
program  contracts.  Following  the  conclusions,  recommendations  will  be  presented. 

Conclusions 

As  discussed  in  Chapter  I,  the  specific  problem  regarding  the  establishment  of 
cost  and  schedule  variance  thresholds  centers  around  the  fact  that  there  is  no  universal 
agreement  on  what  constitutes  appropriate  threshold  levels.  Different  interpretations  of 
variance  thresholds  between  government  and  defense  contractor  personnel  indicated 
the  need  for  setting  appropriate  variance  thresholds.  Research  presented  in  Chapter  II 
demonstrated  that  there  were  many  well  developed  theoretical  models  to  help  establish 
such  threshold  levels.  Although  variance  thresholds  are  found  in  many  defense 
acquisition  contracts,  little  was  known  about  the  methods  the  government  and 
contractors  use  to  establish  these  threshold  values. 

As  the  results  in  Chapter  IV  indicated,  DoD  and  defense  contractor  personnel 
rely  on  personal  management  experience  and  Judgment  in  establishing  the  variance 
threshold  levels  for  a  contract.  One  possible  explanation  for  this  occurrence  is  the 
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apparent  lack  of  awareness  of  the  many  theoretical  models  available  which  can  be 
applied  to  the  establishment  of  variance  thresholds  for  defense  contracts.  This  lack  of 
knowledge  could  have  been  the  result  of  inadequate  supplemental  training  of  personnel 
which  could  expose  them  to  these  additional  methods. 

In  addition,  government  documents  and  regulations  as  well  as  defense 
contractor  documents  did  not  provide  any  specific  methods  or  techniques  to  derive 
cost  and  schedule  variance  thresholds.  This  lack  of  specific  guidance  funher 
contributes  to  the  necessary  reliance  on  personal  experience  and  judgment  to  set  the 
threshold  levels. 

Recommendations 

Upper  management  should  provide  functional  personnel  with  additional 
supplemental  training  to  expose  them  to  the  various  methods  to  establish  variance 
thresholds  which  can  be  applied  to  DoD  major  program  contracts. 

This  thesis  has  shown  how  DoD  establishes  cost  and  schedule  variance 
thresholds.  But  this  research  has  only  laid  the  foundation  for  further  investigation  into 
this  subject.  A  study  to  determine  the  relative  effectiveness  of  variance  threshold 
levels  for  all  DoD  major  program  contracts  should  be  conducted.  Such  a  study  would 
aid  the  decision  maker  as  to  whether  any  changes  should  be  made  to  improve  the 
effectiveness  of  the  current  practices  used  to  establish  variance  thresholds. 

Further  study  should  be  performed  to  determine  specific  guidelines  for 
establishing  variance  thresholds.  This  study  should  be  based  on  available  empirical 
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data  from  past  and  on-going  programs.  Informal  techniques  which  are  intuitive  in 
nature  should  not  be  the  only  tool  used  to  establish  variance  thresholds. 

Summary 

This  chapter  provided  some  insights  into  the  current  practice  by  government 
and  defense  contractor  personnel  with  regard  to  the  establishment  of  variance 
thresholds.  It  also  identified  the  lack  of  awareness  of  the  various  methods  of 
establishing  variance  thresholds  which  could  be  effectively  applied  to  DoD  major 
program  contracts.  Finally,  recommendations  were  made  which  address  increased 
training  requirements  and  potential  follow-on  studies  to  determine  the  relative 
effectiveness  of  variance  threshold  levels  and  finally  achieve  success  in  controlling 
cost  overruns  problems. 
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Appendix  A:  Listing  of  Major  Defense  Acquisition  Programs 
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THE  UNCER  SECaSTARY  Of  OEr  E.NSS 
WASHINGTON.  OC  20301 

acquisition  ^3  P£3  .5^ 

MZMCRANDCM  for  SiCRZTARIZo  OF  TEZ  HZLZT.KP.'i  DEPARTMENTS 
SUBJECT:  Desisr.aticn  of  Major  Deier.se  Accuisiricn  ?rograr.s 


Attac.".ed  are  updated  listincs  of  Defe-'ise  Acquisition 
Board  (DAB)  and  Cc.'r.pcne.rt  programs.  These  listings 
officially  update  t.he  February  23,  1389,  versions. 

The  DAB  and  Component  categories  are  used  to  determine 
whether  the  programs  are  subject  to-  nanacement  oversight 
either  by  the  DAS  or  by  the  Component  decision  prccess. 

These  listings  do  not  reflect  or  obviate  separate  guidance  cn 


Attachments 

cc : 

DAS  Principals  s  Advisors 
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Asof.lanniu  y  20, 1000 


Appendix  B:  Interview  Summaries 
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Government  A,  USAF,  MAJCOM  Staff 


Date:  18  Jan  93 
Position:  Staff 
Summary: 

-  This  office  does  not  set  variance  thresholds  for  SPOs 

-  Gov’t  sets  external  variance  thresholds 

-  Contractor’s  set  internal  variance  thresholds 

-  Internal  tighter  than  external 

-  Staff  tries  to  educate  folks  through  trainii;g  and  memos/policy  letter(s) 

-  Disagrees  with  analysis  is  excessive  as  often  claimed  by  contractors 

-  Cost  is  not  an  issue/factor 

-  Variance  threshold  levels  depend  on  PM  interest  in  variance  analysis 

-  Experience  that  variance  thresholds  set  using  whatever  is  in  last  contract 

-  Sometimes  top  10  cost  drivers  are  chosen  (WBS  $wise  or  WBSs  which  may  be 
troublesome  areas) 
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Government  B,  USAF,  Systems  Program  Office 
Date:  9  Feb  93 

Position:  Program  Financial  Manager 
Summary: 

-  A  subjective  process 

—  If  no  problem  perceived  then  reuse  old  variance  thresholds  from  previous 
contract(s) 

-  Same  variance  thresholds  for  all  (i.e.  cumulative  and  current  month) 

-  Troubled  that  establisher  and  users  are  not  the  same  (i.e.  cost  estimator  versus  budget 
executioner) 

-  User  of  variance  thresholds  analysis  develop  attitudes  toward 

-  Variance  thresholds  are  +/-  10%  or  $50,000  of  BCWS  and  BCWP  whichever  is 
greater  and  10%  for  BAG  in  any  reported  WBS  element 
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Government  C,  USAF,  Systems  Program  Office 
Date:  2  Apr  93 

Position:  Financial  Manager  for  engine  contract 
Summary: 

-  Contract  phase  is  HMD 

—  Contract  let  in  Aug  19xx 
--  Duration  is  19xx-2xxx 

-  Plan  to  produce  9  a/c  and  27  engines  for  HMD,  plan  procure  648  a/c. 

-  $1.5B  contract 

-  Cost  plus  award  fee  contract 

—  Cost  +4%  base  fee  (guaranteed)  and  9%  incentive  fees) 

-  Contract  recently  rephased  stretching  development  1 .5  yrs 

-  First  flight  expected  summer  I9xx 

-  FM  has  6  financial  analysts 

-  Yes,  individual  established  variance  thresholds  for  this  contract 

-  Past  experience  used  to  establish  variance  thresholds. 

-  Not  aware  of  other  theoretical  models 

-  Factors 

—  Prep  time  does  not  play  here 
—  Risk  is  not  a  factor 

—  Phase  is  factor  in  setting  variance  thresholds 
—  For  current  month  value 

—  Variance  thresholds  stay  the  same  throughout  the  contract 
—  For  cumulative  amounts 
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—  May  vary  throughout  the  life  of  the  contract 

—  Vt  usually  gets  tighter  toward  the  end. 


-  Quit  reporting  at  90-95%  spent 

-  Yes  cost  is  a  consideration 
—  No  other  factors  considered 

-  Variance  thresholds  for  the  contract  are  10%  and  $250K  (cumulative) 

-  Variance  thresholds  are  negotiated  with  contractor 

-  Contractor  internal  variance  thresholds  is  5%  and  $50K 

-  Contractor  generates  and  gives  individual  flash  report  Format  1  (raw  data)  5  to  10 
days  after  the  close  of  each  acct  period 

—  Individual  does  a  quick  analysis  for  internal  use 

-  SPO  conducts  a  monthly  review  on  program  status  with  the  conu’actors  paying 
special  attention  9  major  WBS  and  those  that  busted  variance  thresholds 

-  Formal  CPRs  are  prepared  but.  .  . 

-  Traditionally  takes  30-45  cl*.ys  to  receive  CPRs  after  the  end  of  acct  period 
which  is  the  fifth  work  day  of  the  month 
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Government  D,  USA,  Systems  Program  Office 
Date:  12  Apr  93 

Position:  Program  Cost  Management  Supervisor 
Summary: 

-  Method  has  recendy  been  changed 

—  Previously  used  a  +/-  $  thresholds  for  1st,  2nd,  and  3rd  level  format 

—  Determined  this  was  not  working  well,  no  way  to  access  if  ’adequate’  thresholds 
were  being  applied 

—  Next  tried  $  thresholds  PLUS  gov’t  contractually  reserved  the  right  to  change 
thresholds  once  it  was  determined  ’too  much/too  little’  reporting 

—  No  contract  modification  necessary  as  this  was  written  into  the  contract 

-  On  the  program 

—  Absolute  value  of  TOP  TEN  variances  by  value  (current  cost  and  schedule, 
cumulative  cost  and  schedule,  and  at  completion  variances) 

—  Which  ones  exceed  10% 

—  If  >  10%  then  report 

—  If  <  10%  do  not  report 

--  This  method  was  derived  from  the  Joint  DoD  Industry  TQM  Team  Report  for 
Program  Management  on  the  Cost  and  Schedule  Management  Process,  May  17,  1991 

—  Noted  that  OSD  is  familiar  with  this  topic 

-  It  is  written  into  the  contract  to  change  thresholds  as  required 

-  There  were  NO  specific  factors  (cost,  risk,  program  phase)  considered  when  setting 
these  thresholds 
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Governmeiii  E,  USN,  Systems  Program  Office: 

Date:  21  Apr  93 
Position:  Staff 
Summary: 

-  Variance  thresholds  not  viewed  a  structured  set  of  values 

—  Viewed  as  a  philosophical  basis  or  a  personal  angle 

-  Depends  on  who  uses  the  info  generated  by  5  C/SCSC  data  elements  (USDA,  SAE, 
PEO,  PM,  or  analyst) 

—  Each  has  different  view  about  variance  thresholds 
-Historical  overview 

-  19xx  SAR  annual  requirements 

—  EAC  from  program  office  and  contractors 

-  Nunn-McCuny  requirements 

—  15%  threshold  overrun  in  program  or  a  major  contract  in  program 

10  days  for  PM  to  report  to  OSD 
—  25  days  to  Congress 

—  25% 

- —  Stop  work 
—  Report  to  Congress 

—  Services  want  an  internal  mechanism  to  respond  to  above  requirements 
—  That  is  they  set  a  tighter  vt 

-  In  late  1970,  USAF  implemented  PAR/CAR/SPR  approach 

—  Program  Acquisition  Approach  (PAR)  with  vt  of  5% 

—  Component  Acquisition  Report  (CAR)  with  vt  of  10% 

—  Secretarial  Acquisition  Report  (SPR)  with  vt  of  15% 
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— -  Red  =  10%  and  over 
Yellow  =  5  to  9.99% 

- —  Green  =  0  to  4.99% 

-  SAR  phased  out  in  1985 

—  Supplemental  Contractor  Cost  Report  (SCCR) 

-  5000.2M,  Part  16,  Section  H  changes  SCCR  to  Supplemental  Contractor 
Cost  Info  (SCCI)  in  1989  (see  p  16-H-l-l) 

-  Naval  Air  Command  comfortable  with  5%  and  10% 

--  Varies  from  location  to  location 

-  Bottom  line 

--  Purpose  of  variance  thresholds 
—  To  sort  out  info 

—  To  prioritize  activities  in  managing  program 
—  To  draw  manager  attention  to  problem  areas 

-  That  is  variance  thresholds  need  no  brain 
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Defense  Contractor  X,  Program  Office: 

Date:  12  Apr  93 

Position:  Subcontracts  Manager,  C/SCSC  monitor 
Summary: 

-  Variance  thresholds  are  established  on  a  ’contract  by  contract’  basis 

—  Vary  by  type  (e.g.  development  vs  production) 

-  Variance  thresholds  may  be  changed  over  time  (contract  phase)  via  re-negotiation 

-  70%  of  contract  C/SCSC  monitoring  is  done  by  subcontractors 
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Appendix  C:  HQ  AFSCIFMC  Policy  Letter 
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FHC 


to 

An* 


DEPARTMENT  OF  THE  AIR  FORCE 

MtAoauAflTSfla  AencNAuTicAt  systems  oivisicn  (afscj 
W«lG>HT-P  A'TTS.'^SON  AIR  FQRCS  EASE.  OHIO  43A3i-460a 


1 7  MAR  iSSZ 


Sample  Financial  Contract  Data  Requirements  Lists  (CDRLs)  and  Lessons  Learned 
for  Preparing  Financial  CDRLs 


,0,  ASD/AE?  ASD/VC? 

MAP  VF? 
RVP  VJ? 
SDP  VL? 


ASD/:a?  ASO/YS? 
YC?  TT? 

YF?  'fVP 

■f??  YZ? 


VL/FH 

OL-FH 

4950TV/AMP 


1.  Several  months  ago  ve  asked  you  for  sample  financial  CDRLs  and  lessons 
learned  concerning  preparation  of  financial  CDRLs.  Ve  consolidated  your 
responses,  Inputs  from  other  product  divisions,  and  the  experiences  fron  our 
staff  cost  analysts. 


2.  Attachment  1  is  a  summary  of  the  lessons  learned.  This  narrative  seir/es  as 
a  reminder  that  certain  aspects  of  the  CDRLs  must  be  negotiated  and  specified  in 
each  RPP/contract .  In  addition,  ve  suggest  approaches  for  establishing 
reporting  levels,  variance  thresholds,  etc.,  vith  explanations  as  to  vhy  the 
approach  is  recomraened. 

3.  Attachment  2  is  set  of  sample  CDRLs  for  the  folloving  reports: 

a.  Contract  Funds  Status  Report  (CFSR) 

b.  Contract  Vork.  Breakdovn  Structure  (CV3S) 

c.  Cost  Performance  Report  (C?R)  -  top  "driver"  variance  approach 

d.  CPR  -  dollar/percentage  variance  approach 

e.  Cost/Schedule  Status  Report  (C/SSR)  -  top  "driver"  variance  approach 

f.  C/SSR  -  dollar/percentage  variance  approach 

Note  that  the  sample  CDRLs  do  not  contain  every  condition  discussed  in  the 
lessons  learned.  Together,  hovever,  the  tvo  documents  should  assist  you  in 
preparing  comprehensive  CDRLs  early  in  the  procurement  cycle  vhich  mitigate  the 
need  for  changes  and  contractual  disputes  later  in  the  program. 

4.  Ve  are  constantly  looking  for  vays  to  improve  our  cost  reporting.  Please 
encourage  your  personnel  to  share  their  knovledge  and  concerns  vith  us,  so  ve 
can  pass  fresh  ideas  on  to  others  and  help  resolve  recurring  problems. 

5.  Please  contact  Capt  Lisa  Hendel,  ASD/FHCA,  55904  vith  comments  or  questions. 

DONMA  J.  VOGc'y 
Director  of  Cost 
DCC.  riiiancial  Management 
ana  Comptroller 


2  Atch 

1.  Lessons  Learned 

2.  Sample  CDRLs 
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L2SSCNS  L£A5ilED  FOR  ?a£?.\RlNG  FINA-SCIAL  CDRLS 


1.  General  Comiaenra 

a.  AL7A7S  5tnov  vhy  you  are  buying  each  financial  CDRL.  Be  able  to  justify 
the  reporting  level,  frequency  of  submission,  distribution  list,  tailoring, 
etc.  Knov  up  front  vho  is  supposed  to  use  the  data  and  make  sure  each  user 
has  input  into  the  CDRL. 

b.  Before  preparing  a  DD  Pora  1423,  carefully  read  the  applicable  Data 
Item  Description  (DID).  Many  of  the  financial  DIDs  require  you  to  specify 
values  in  individual  CDRLS  (these  aay  require  negotiation).  These  value J'- 
include  variance  thresholds,  forecast  periods,  etc.  Not  all  saaple  CDP-Ls  vill 
include  these  areas,  so  use  the  DID  as  your  primary  source. 

c.  Cross-check  your  DD  Pora  14223  vith  the  content  of  the  Contractor  Cost 
Data  Reporting  (CCDR)  Plan.  The  CCDR  Plan  included  in  the  solicitation  and 
contract  contains  the  summary/ reporti.ng  level  Contract  Vork  Breakdovn 
Structure  (C«BS)  and  the  types,  frequency,  and  reporting  levels  of  the  major 
financial  CDRLs. 

d.  Incorporate  revisions  to  t.ne  sunuaary  CV3S  and  financial  CDRLs  due  to 
proposed  contract  changes  before  the  changes  are  sent  to  the  contractor. 

e.  In  the  past,  some  CDRLs  have  included  information  pertaining  to  timing 
of  Cost/Schedule  Control  Systems  Criteria  (C/SCSC)  implementation,  estimate  at 
completion  (EAC)  frequency,  delivery  of  system  descriptions,  flovdcvn  of 
C/SCSC  and  Cost/Schedule  Status  Report  (C/SSR)  to  subcontractors,  restrictions 
on  handling  of  material  costs,  etc.  These  areas  should  be  handled  as  special 
contract  provisions  as  they  pertain  to  the  cost/schedule  management  clauses 
(C/SCSC  or  C/SSR)  contained  in  the  solicitation  and  contract. 

f.  Accept  contractor  format  vnenever  possible  to  reduce  data  costs. 

g.  Use  electronic  transfer  and  split'deliyefy.-'dates- to  cut*  transaission 

time.  Use  of  the- ANSIjASC  Jll  Electronic..  Data . •  Interchange  standards- is  nov 
mandatory  for  transmission  of  cost  aanageaent  information.  '  '  "  " 

h.  Be  extremely  clear  on  the  delivery  requirements.  For  «ajq>le,.  you. 
may  assume^  that  delivery  of  the  first  Contract.  Pund^- Status- Report  HLtrlSi:-' 
calendar  days  after  the  end  of  the  first  Government'  fiscal  quarter  follovlng 
contract  avard  includes  partial  fiscal  quarters,  but.  Che  contractor  say  -- 
interpret  "first"  to  mean  the  first  full  Government  fiscal  quarter. 

i.  Strategize  hov  you  vant  the  contractor  to  handle  special  aspects  of 
the  contract,  such  as  avard  fee,  long  lead  items,  mi.xed  contract  types,  etc. 
Separate  CPRs  are  required  for  R&D  and  production.  Separate  CFSR  pages  are 
required  for  each  contract  appropriation  but  are  bought  on  the  same  CDRL. 

j.  Request  a  copy  of  the  contractor's  accounting  calendar  so  you  knov 
vhen  each  monthly  accounting  period  closes  and  can  track  CDRL  due  dates. 

k.  Include  a  copy  for  ASD/FMCA  on  the  distribution  list  of  all  fi-uancial 
CDRLs.  Ve  vill  for.'ard  this  copy  to  the  Cost  Library  for  permanent  storage. 
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Concrac:  Funds  Stacus  Rasort  (C~;?.) 


a.  Structure  reporting  levels  ::r  31sc!<  11  by  total  contract  vhenever 
possible.  Slock.  11  provides  obligation  and  expenditure  inforaaticn  to  support 
contract  funding  actions,  includi.ng  overrun  and  underrun  adjustaents.  If  the 
contracting  officer  is  funding  at  total  contract  and  vould  aalte  all  billing 
price  adjustae.nts  at  total  contract,  t.Hen  31ocic  11  reporting  should  be 
specified  as  total  contract.  If  funding/billing  price  adjustaents  sia/  be  done 
by  separately  priced  Contract  Line  'tea  Nuaber  (CLI.N)/3ub-CLI.N,  then  31odc  11 
reporting  eleaencs  should  be  specified  as  separately  priced  CLINs/subCLIHa.  If 
you  have  efforts  on  the  same  contract  funded  only  by  specific  fund  cites  (A7 
versus  SDIO,  "h"  account  versus  current  year,  etc.),  you  may  have  to  require 
separate  C?SR  pages  for  each  effort  to  have  tiae-phased  obligation  and 
e.xpenditura  forecast  data  for  each  effort  from- Blocisr  12-14. 

b.  Tou  Bust  reconcile  the  actual  and  at  completion  CFSR  data  vith  the  C?R 
or  C/SSR.  Standard  adjustments  include  fee/profit,  differences  in  handling  of 
material  and  subcontractor  coats,  etc.  To  assist  you,  request  the  contractor 
attach  a  reconciliation  to  the  C?S?..  Contact  the  I??.0  or  DCAA  for  assistance 
in  performing  the  reconciliation. 

c.  The  forecast  periods  of  open  coaaitments,  accrued  expendituraSr 

billings,  and  termination  liability  in  Blocks  12-14  must  be  specified  in  the 
CDRL.  Detailed  data  is  needed  for  incremental  funding  decisions  in  the 
current  fiscal  year,  vith  some  data  .needed  to  support  the  ne.xt  fiscal  year's 
forecast.  Consider  using  the  folloving  forecast  parameters:  monthly  for  the 
current  fiscal  year,  quarterly  for  at  least  four  quarters,  and  then  quarterly, 
semi-annually,  or  annually  to  completion.  The  contractor  can  continue  the 
forecast  on  an  attachment  if  it  exceeds  the  number  of  forecast  brocks  printed  / 
on  the  DD  Form  1586  or  continue  on  a  second  DD  Form  1586.  ' 

d.  Blocics  12-14  are  projectiohs'of  authorised  vorSc  only r- matching  the., 
subtotal  in  Block  llg.  It  does  not  include  vorlc  vhich  has  not  been  authorized 
(options,  change  orders,  supplemental  agreements,  etc.) -from  Blocks  llh-llj. 

If  you  plan  to  authorize  nev  vork  in  the  current  fiscal  year,  make  sura  you 
get  an  updated'funding  profile  froa'the  contractor' vnlcS'f Includes.. bo tifTthe -'old 
and  nev  efforts  prior  to  authorization- of  the^nev.  yorki^>-^  -  .  r^r_ 

e.  The- CFSR  data  (especially  the- forecast,  of  accrued^ expendi turesr  aadj 
terainatioQ  liability)  should  supp-orfcontractor's’  requests-  for-  incrasiefttal;. 
fimdlng..\ir.der  the  Limitation  of  Funds;  (LOP)  clauser '.loO-can  require*-th«5x^h; 
contractor  to  include  a  projected  date  for  submission- of  his  next  LO?  ' 
notification  in  the  Remarks  section  (Block  15)' so  you  can  reconcile  that  date 
against  the  numbers  in  the  CFSR.  The  contracting  officer- can- provide.- tharr 
exact  LOF  parameters  (i.e.,  notification  due  60  days  prior  to  75Z  expended,  60 
days  prior  to  lOOX  expended,  etc.)  foe  each  contract. 
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3.  Contrac:  Voric  Breakdovn  Structure  (GTBS) 


a.  Include  MIL-STD-381  (latest  version)  in  your  solicitation  and  contract 
as  a  coopliance  document. 

b.  Reporting  levels  for  the  CTBS  shall  reference  the  CCDR  Plan  attached 
in  the  solicitation  and  contract  (separate  attachment  or  attached  to  this 
CDRL) . 


c.  Tou  need  to  specify  the  level  to  vhlch  the  contractor  extends  the 
summary  CTBS  provided  in  the  CCDR  Plan  to  fora  the  complete  CVBS.  For 
contracts  vith  C/SCSC,  request  extension  to  the  cost  account  level.  For  C/SSR 
or  CPR  (vithout  application  of  C/SCSC)  contracts,  request  extension  to  the- 
level  where  the  work  will  be  done/costs  accumulated.  Require  identification 
of  subcontracted  effort  by  subcontractor  name  at  the  lowest  level  of  the'C73S. 

d.  Part  I  of  the  CTBS  is  the  Index,  which  outlines  the  elements  of  the 
extended  CVBS  and  includes  a  cross-reference  to  the  CLINs,  Statement  of  Vorx 
(SOW),  and  specifications.  If  the  CVBS  was  properly  used  to  structure  the 
SOW,  CLINs,  and  specifications,  the  cross-reference  should  be  simple.  If  the 
CVBS  wasn't  used  to  structure  the  ocher  areas,  you  should  attached  a 
cross-reference  at  the  summary  CVBS  level  to  the  CCDR  Plan  to  assist  the 
contractor  in  performing  Che  cross-reference. 

e.  Part  II  of  the  CVBS  is  a  narrative  explanation  of  the  extended  CVBS 
elements.  Since  reporting  is  done  at  the  smmar/  level,  the  dictionary  is 
helpful  in  identifying  the  detailed  content  of  each  summary  reporting  level. 
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4.  Cost  PerfocT.anca  Report  (C?H) 


a.  Foraac  1  raportioj  elaments 
the  solicitation  and  contract. 


s'r.ould  caferenca  t.ne  CCCR  Plar.  a 


ached 


in 


b.  To  iaprove  nonthly  analysis  and  ZAC  seneraticn,  require  Fornat  1  C«3S 
elanents  to  contain  only  direct  costs,  as  these  costs  are  under  the  program 
oanager's  control.  Overhead,  General  and  Adainistrative,  and  Cost  of  Money 
should  be  reported  and  analysed  separately  since  they  are  iapacted  by  facility 
changes  as  veil  as  program  actions.  If  cost  estiaating  personnel  require 
fully-burdened  CVBS  data  for  their  purposes,  they  should  include  a  CDRL  foe 
the  Cost  Data  Suamary  Report  (annual  submission  to  support  the  annual  program 
estimate  or  a  single  delivery  at  contract  completion).  The  C?.R  is  structured 
to  support  performance  measurement,  not  cost  estimating. 

c.  Specify  Format  2  reporting  elements,  normally  the  second  level  of  the 
contractor's  organisation  structure  (Z.ngineering,  Hanufacturl.ng,  etc.). 

d.  If  you  have  critical  subcontractors,  you  can  have  each  subcontractor 
reported  separately  on  Format  2,  rather  than  ccmbi.ned  vith  all  material  costs. 

e.  For  the  largest  contracts,  consider  having  the  Level  2  organisational 
elements  Include  direct  labor  costs  only.  Material,  subcontracts,  other 
direct  costs,  and  indirect  costs  vouid  be  reported  separately.  Segregation  of 
direct  labor  cost  by  organisation  allovs  you  to  compare  actual  a.nd  forecasted 
labor  dollars  on  Format  2  vith  actual  and  forecasted  manpover  on  Format  4. 


f.  If  there  is  C/SCSC  and/or  C/SSR  flovdovn  to  subcontractors,  request 
the  contractor  include  a  summary  of  subcontractor  performance  measurement  data 
vith  his  C?R.  Attachi.ng  the  subcontractor  C?R  or  C/SSR  is  acceptable. 

g.  Specify  t.hat  t.he  reporting  elements  for  labor  vill  be  the  same  for 

Format  2  and  Format  4.  ’ 


h.  You  must  specify 
that  the  contractor  must 


the  forecast  periods  for  Format  3  and  Format  4  and 
use  the  same  forecast  periods  for  both  formats. 


i.  There  are  tvo  approaches  for  Format  1  and  Format  2  variance  analysis 
thresholds; 


.  •(!)  The  preferred  method  is  to  concentrate  on  those  CV3S  elements 

vhich  have  the  largest  impact  on  cost  and  schedule  performance.  Under^  the- 
"top  driver"  method,  the  contractor  prioritises  the  CVBS  elements  in  terms  of 
cost  driver  status  and  schedule  driver  status.  The- Government  approves  the 
lists.  The  list  is  updated  to  reflect  changes  during  the  contract. 

(2)  The  contractor  submits  current  month  and  cumulative  cost  and 
schedule  variaince  analysis  for  the  top  50Z  of  the  CT3S  elements  from  the 
approved  driver  lists.  You  can  specify  a  minimum  dollar  threshold  belov  vhich 
the  contractor  does  not  submit  vritten  analysis. 

(3)  Include  variance  thresholds  for  the  indirect  cost  elements. 

(4)  All  variances  at  completion  exceeding  -r/-  5Z  and  a  minimum 
dollar  value  are  explained. 
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(5)  The  3'ar.dard  variance  analysis  approach  is  to  coabine  dollar  and 
percentage  thresholds.  Vhen  an  eleaent  breaches  both  thresholds,  vritten 
analysis  is  provided.  The  difficulty  is  that  each  G.’BS  and  indirect  cost 
eieaenc  has  varying  dollar  values.  Choosing  one  dollar  threshold  vhich 
provides  the  opci.auni  aaounC  of  analysis  is  difficult.  Tou  can  specify 
different  thresholds  for  various  eleaents  but  this  increases  the  contractor's 
cost. 


(6)  If  you  choose  the  standard  aethod,  you  should  change  thresholds 
during  the  life  of  the  contract  for  the  cumulative  ‘variances.  Recoanend  these 
thresholds  c.hange  vhen  the  total  contract  reaches  a  certain  percent  coaplatt- 
rather  than  after  a  certain  number  of  aonths  (in  case  the  program  is  slipping 
behind  schedule).  The  cumulative  dollar  thresholds  can  be  raised  over  tiae 
vhile  tighteni.ng  the  percentage  thresholds. 

(7)  Current  aonth  variances  can  use  one  threshold  for  the  life  of  the 
contract. 

(8)  Variances  at  completion  can  also  use  a  single  threshold,  since 
you  vant  analysis  on  those  variances  vhich  the  contractor  cannot  resolve 
before  contract  completion. 

j.  Tou  must  define  the  "significant  changes"  in  Format  3  vhich  require 
e.xplanation  in  Format  5.  Thresholds  should  included  changes  i.n  each  forecast 
period  and  the  total  performance  measurement  baseline-  Use  a  percentage 
threshold,  such  as  +/-  lOX,  or  combine  it  vith  a  minimum  dollar  value. 

k.  Tou  must  define  the  "significant  changes"  in  Format  4  vhich  require 
e.xplanation  in  Format  5.  Thresholds  apply  to  current  month  actual  manpover, 
forecasted  tnanpover  by  period,  and  total  manpover  at  completion.  Consider 
using  a  percentage  threshold,  such  as  +/-  lOt,  since  the  amount  of  manpover 
varies  greatly  betveen  functional  categories.  Tou  must  also  specify  vhether 
the  variances  are  calculated  for  each  reporting  element  or  total  contract. 

l.  Tailor  Format  4  to  provide  the  labor  budget  baseline  (extracted  from 
Format  3)  in  the  same  units  as  the  rest  of  Format  4.  Without  this  labor 
budget  baseline,  you  cannot  tell  vhether  changes  in  aanpover.  art  related  to 
increasi.ng  the  baseline  effort,  increasing  the  manpover.  estimate,,  or  both.. 

B.  Vhen  creating  thresholds,  reaeaber  that  -15Z  is  unsatisfactory  vhen>” 
reporting  to  higher  headquarters.  Tou  should  receive  data  fzoB.--tbe -contractor 
vhich  is  at  least  as  stringent  as  this  definition.  • 

n.  Include  a  provision  for  periodic  reviev  of  variance  thresholds  to 
ensure  the  initial  thresholds  are  proving  the  right  amount  of  data.  Changes 
should  be  negotiated  and  a  nev  CDRL  included  via  contract  modification. 
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5.  Cosr/Schedule  Status  Report  (C/SSRI 


a.  C'-BS  repoc'ing  eleaenis  should  reference  the  CCDR  Plan  attached  in  the 
solicitation  and  contract. 

b.  To  improve  monthly  analysis  and  EAC  generation,  require  the  CT3S 
elements  to  contain  only  direct  costs,  as  these  costs  are  under  the  prograa 
manager's  control.  Overhead,  General  and  Administrative,  and  Cost  of  Honey 
should  be  reported  and  analysed  separately  since  they  are  impacted  by  facility 

c.hanges  as  veil  as  program  actions.  If  cost  estimating  personnel  require 
fully-burdened  CV3S  data  for  their  purposes,  they  should  include  a  CDRL  for 
the  Cost  Data  Summary  Report  (annual  submission  to  support  the  annual  progra* 
estimate  or  a  single  delivery  at  contract  completion).  The  C/SSR  is 
structured  to  support  performance  aeasurement,  not  cost' estiaati.ng. . 

c.  If  there  is  C/SSR  flovdovn  to  subcontractors,  request  the  contractor 
include  a  summary  of  subcontractor  performance  measurement  data  vith  his 
C/SSR.  Attaching  the  subcontractor  C/SSR  is  acceptable. 

d.  There  are  tvo  approaches  regarding  variance  analysis  thresholds: 

(1)  The  preferred  method  is  to  concentrate  on  those  CV3S  eleaants 
vhich  have  the  largest  Impact  on  cost  and  schedule  performance.  Under  the 
"top  driver"  method,  the  contractor  prioritizes  the  CVBS  elements  in  terms  of 
cost  driver  status  and  schedule  driver  status.  The  Government  j'-'prcves  the 
lists.  The  list  is  updated  to  reflect  changes  during  the  contract. 

(2)  The  contractor  submits  current  month  and  cumulative  cost  and 
schedule  variance  analysis  for  the  top  SOX  of  the  CVBS  elements  from  the 
approved  driver  lists.  You  can  specify  a  minimum  dollar  threshold  belov  vhich 
Che  contractor  does  not  submit  written  analysis. 

(3)  Include  variance  thresholds  for  the  indirect  cosr  elements/""  ’ 

(4)  All  variances  at  completion  exceeding  +/-..5X.  and  a  miaiaua. _ _ 

dollar  value  are  explained.-  ' 

(5)  The  standard  variance  analysis  approach  is  to  combine  dollar,  and 

percentage  thresholds.  Vhen  an  element  breaches- both  thresholds,  vritteasr'- 
analysis  is  provided.  The  difficulty  Is  that  each  CVBS, and  Indirect  coat;_ 
element  has.  varying  dollar  values.-  Choosing-  one-  dollar:  threshold  vhich.-. l,'^: 
provides  the  optimum  amount  of  analysis  is  difficult.  .  - 

(6)  If  you  choose  the  standard  method,  you  should  change  thresholds 
during  the  life  of  the  contract.  Recommend  thresholds  change  vhen  the  total, 
contract  reaches  a  certain  percent  complete  rather  than  after  a  certain  number 
of  months  (in  case  the  program  is  slipping  behind  schedule).  The  dollar 
thresholds  can  be  raised  over  time  vhile  tightening  the  percentage  thresholds. 

(8)  Variances  at  completion  can  use  a  single  threshold,  as  you  need 
analysis  on  variances  the  contractor  cannot  resolve  before  completion. 
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«.  Vhen  creating  threaholda,  reaeaber  that  -15J  is  unsatisfactor/  vhen 
reporting  to  higher  headquarters.  Tou  should  receive  data  frca  the  contracto 
which  is  at  least  as  stringent  as  this  definition. 

f.  Include  a  provision  for  periodic  review  of  variance  thresholds  to 
ensure  the  initial  thresholds  are  proving  the  right  ainount  of  data.  Changes 
should  be  negotiated  and  a  new  CDRL  included  via  contract  nodification. 


SA*MPLZ  COffTSACr  FUNDS  STATUS  R2PORT  (CPSR)  CDPi, 

1.  SEGUZ.MCi  NUHBER:  AQOX 

2.  TITLE:  CONTRACT  FUimS  STATUS  SSPORT  (C?SR) 

3.  SUBTITLE: 

4.  AUTEORITT;  DI-F-SGOAB/T 

Report  at  price  (including  fee/profit)  by  separately  priced  Contract  Line  Itea 
Number  (CLIN)  or  subCLIN  and  Total  in  Blocks  lla-llm.  Projections  in  Blocks 
12-14  shall  be  monthly  for  the  duration  of  the  current  fiscal  year,  quarterly 
for  at  least  four  quarters,  and  Chen  quarterly,  semi-annually,  or  annually  to 
completion.  As  part  of  Block  15  (Remarks),  contractor  shall  include  a  projected 
date  for  the  next  vricten  notification  required  by  the  "Limitation  of  Funds" 
clause.  Contractor  shall  include  a  reconciliation  of  the  actual  to  data  and  at 
completion  data  reported  in  the  CPSR  vith  the  Cost  Performance  Report  or 
Cost/Schedule  Status  Report  for  the  same  accounting  period.  Contractor  format 
acceptable  if  it  includes  all  required  data  and  is  approved  by  the  Government. 


5. 

CONTRACT  R2FERENC3;  'SOV  TASR  I.X.X 

6. 

TECENICAL  OFFICE:  ASD/XXPP 

7, 

DD  250  REQ:  LT 

8. 

APP  CODE; 

9. 

INPUT  TO  lAC; 

10. 

FREQUENCY ;  QTRLT 

.  ■ ■ 

11. 

AS  OF  DATS;  Last  day  of  the  contractor's 

monthly  accounting  period 

nearest  the  end  of  the  Mar,  Jun,  Sep,  and  Dec  Government  fiscal  quarters. 

12.  DATS  OP  1ST  SUBHISSION;  Due  NLT  20  calendar  days  after  the  end  of  the 
Government  fiscal  quarter  (including  partial  quarter)  subsequent  to  the 
contracting  officer's  authority  to  proceed  (including  undefinitized  contracting 
actions). 

13.  '  DATS  OP  SUBSSQUBNT  SUBHISSION/SVBNT  ID:  Due  NLT  20  calendar  days  after  the 
end  of  each  Government  fiscal  quarter.  CFSE  data  shall  be  reconciled  to  the 

30  Sep  Government  fiscal  year  end  if  the  contractor's  accounting  period  does  not 
end  on  30  Sep . 


14. 

DISTRIBUTION: 

ASD/XXXI 

ASD/XXPP 

ASD/FMCA 

15, 

TOTAL: 

16. 

REHAPvKS: 
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SAMPLZ  CONTIlACr  VORX  3R2AKDOVN  STIIUCTURS  (C/BS)  C2RL 

1.  SiQUI.S'CS  NUMBER:  ACOX 

2.  TITLE:  CONTRACT  VORX  BRZAiOXlW  STRUCTURS  (CV3S) 

3.  SUBTITLE: 

4.  AUTHCRIT7:  DI-A-3Q23/T 

The  eras  In<ie:«  (Part  I)  and  Dictionar/  (Part  II)  shall  be  prepared  in  accordance 
vith  the  Contractor  Cost  Data  Reporting  Plan  (CCDRP)  attached  to  the  Request  for 
Proposal  (RRP)  and  contract.  Contractor  shall  also  use  mL-STD-881A  (or  latest 
approved  version)  for  guidance  in  preparation  of  the  CraS. 

The  CTBS  Index  and  Dictionary  shall  be  extended  froa  the  stimnar/  reporting  level 
provided  by  the  Government  in  the  CCDRP  to  the  lovest  level  at  vhich  work,  will 
be  done  (the  cost  account  level  for  contracts  vith  a  Cost/Schedule  Control 
System  Criteria  requirement),  including  identification  of  each  individual 
management/ cost  account.  Contractor  shall  identify  subcontracted  effort, 
including  subcontractor  name,  at  the  lovest  level  of  the  extended  CVBS. 

Contractor  format  shall  suffice  if  it  contains  all  required  data  and  is  approved 
by  the  Government. 

5.  C0NTR.\CT  REFERENCE:  SOW  TASK  I.X.X 

6.  TECHNICAL  OFFICE:  ASD/XXP? 

7.  DD  250  REQ:  LT 

8.  APP  CODS:  -  -  - 

9.  INPUT  TO  lAC:  '  ' 

10.  FRSQUENCT:  ONB/R 

11.  AS  OF  DATB:  '  -  ■ 

12.  DATE  OF  1ST  SUBMISSION:  Draft  C7BS  Index  (Part  I)  shall  be  subadtted  vith 
the  proposal  in  response  to  the  Goversaent  RFF. 

13.  DATE  OF  SUBSEQUENT  SUBHISSI0N/E7ENT  ID:  The  cosplete  CWSS  Index  and 
Dictionary  (including  any  changes  agreed  to  during  negotiations)  shall  be 
submitted  NLT  60  calendar  days  after  the  contracting  officer's  authority  to 
proceed  (including  undefini tired  contracting  actions).  A  revised  Index  and 
revisions  to  Dictionary  pages  (due  to  Government-directed  changes  and  internal 
changes)  shall  be  submitted  NLT  30  calendar  days  after  the  CVBS  changes. 


14. 

DISTRIBUTION; 

ASD/XXXX 

ASD/XXPF 

ASD/FMCA 

15. 

TOTAL; 

16. 

REMARKS: 

PCO 

ACO 
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SAhPLZ  COST  PSRPORMAitCZ  RZFORT  CZRL 
(TO?  VARIAMCZ  DRIV'ZR  APPROACH) 

1.  SZa’JENCS  SUy.SER:  ACCX 

2.  TITL2;  COST  PSRPORHANCZ  R£?ORT  (C?R) 

3.  S’JBTITLZ: 

4.  AUTHORITT:  DI-P-6CGOC/T 

Foraats  1-5  apply.  Contractor  foraat  acceptable  if  it  contains  all  required 
data  and  is  approved  by  the  Governnent. 

Poraat  1  Contract  Vork  3rsakdovn  Structure  (C73S)  reporting  elaaents  shall  be  in 
accordance  vith  the  Contractor  Cost  Data  Reporting  Plan  attached  in  the 
contract  and  shall  include  direct  cost  only.  Overhead/fringe  benefits,  General 
and  Adflinistrative  (GoA),  and  Cost  of  Honey  (COM)  shall  be  reported  as  three 
individual  elements  added  to  the  direct  cost  subtotal.  Major/critical 
subcontractor  sumnary  level  perfornance  measurement  data  shall  be  reported  as  an 
attachment  to  Format  1  (subcontractor  C?R  or  C/SSR  is  acceptable). 

All  Budget  at  Completion  (3AC)  changes  on  Formats  1  and  2  shall  be  explained  in 
Format  5.  The  explanation  of  Management  Reserve  forecasted  co.nsuaption  required 
in  Format  5  shall  include  estimated  dollar  value,  potential  vorx  scope,  and 
estimated  budget  distribution  data. 

Reporting  elements  for  Formats  2  and  4  s.hall  be  the  second  level  of  the 
contractor's  organicational  structure  (i.e.,  Engineering,  Manufacturing,  etc.). 

Formats  3  and  4  shall  include  identical  forecast  periods.  These  periods  shall 
be  monthly  for  at  least  six  months,  quarterly  for  at  least  tvo  quarters,  and 
then  quarterly,  semi-annually,  or  annually  to  completion.  Any  c-hange  in  the 
Format  3  total  Performance  Measurement  Baseline  (?H3)  and/or  any  change  in  a 
forecast  period  vhich  exceeds  lOZ  shall  be  e.xplained  in  Format  5.  Any  change  in 
current,  forecast  period,  or  total  manpover  for  each  Foraiat  4  reporting  element 
vhich  e.xceed3  lOZ  shall  be  explained  in  Format  5.  Include  on  Format  4  (am  a 
non-add  item  narked  "BGi’S")  the  total  labor  budget  baseline  spread  (in  the  same 
units  as  the  rest  of  Format  4)  e.xtracted  fron  the  total  budget  baseline  dollars 
on  Format  3. 

On  Foiraat  5  of  the  first  CPR,  the  contractor's  program  manager'  shall  rank,,  in 
descending  order  of  criticality  (i.e.,  the  most  critical  element  at  the  top), 
all  Format  1  CVBS  repotting  elements  anticipated  to  be  schedule  variance 
"drivers."  The  contractor  shall  also  rank,  in  descending  order  of  criticality, 
all  Format  1  CVBS  elements  anticipated  to  be  cost  variance  drivers.  The 
Government  reserves  the  right  to  modify  these  rankings  based  on  its  perception 
of  criticality.  Contractor  shall  submit  updated  schedule  variance  and  cost 
variance  rankings  ever/  six  months,  as  a  minimum,  based  on  performance  to  date 
and  anticipated  problems.  If  the  contractor  uses  critical  path/netvorking 
scheduling  tec-hniques,  identification  of  the  critical  path  and  near  critical 
path  CTBS  elements  shall  meet  the  schedule  variance  driver  requirement. 

Variance  analysis  shall  be  reported  for  each  Format  I  reporting  eleme.nt  as 
follows  (subcontractor  variance  analysis  shall  be  i.ncluded  as  a  part  of  the 
prime  contractor's  variance  analysis): 
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SAMPLZ  COST  P£JIFOP“a:icZ  RSPORT  CDRL  (CONTINUED) 

(TOP  VARIANCE  DRI’VRR  APPROACH) 

a.  Currenc  monch  and  cumulative  schedule  variances  (S'/)  -  SV  narratives  shall 
be.submi tted  for  those  CVBS  elements  vhich  represent  the  top  501  of  the  SV 
drivers  discussed  above,  vith  a  ainiaua  SV  of  $50,000.  If  there  are  fever  than 
10  schedule  variance  drivers,  all  variances  over  $50,000  shall  be  explained. 

b.  Current  month  and  cumulative  cost  variances  (C/)  -  C7  narratives  shall  be 
submitted  for  those  C/BS  elements  vhich  represent  the  top  50Z  of  the  C7  drivers 
discussed  above,  vith  a  minimum  C/  of  $50,000.  If  there  are  fever  than  10  cost 
variance  drivers,  all  variances  over  $50,000  shall  be  explained. 

c.  All  current  and  cumulative  cost  and  schedule  variances  for  the  indirect  cost 
elements  vhich  exceed  +/-  lOZ  and  $50,000  shall,  be  explained. 

d.  All  variances  at  completion  exceeding  +/-  5Z  and  $50,000  shall  be  addressed. 

e.  Specific  corrective  actions,  forecasted  closure  date,  and  impact  to  the 
Estimate  at  Completion  (ZAC)  shall  be  included  in  each  variance  narrative. 
Contractor  shall  go  to  the  lovest  C.'BS  level  (Including  the  cost  account  level) 
needed  to  completely  explain  the  variances. 

f.  Contractor  shall  provide  a  suanacy  variance  analysis  of  Format  2  reporting 
elements  vith  cumulative  schedule  or  cost  varian.es  exceeding  +/-  $100,000. 

g.  Contractor  shall  format  and  submit  the  CPR  in  accordance  vith  the  attached 
specification  for  electronic  data  transmission. 

5.  CONTRACT  REFERENCE:  SOV  TASK  X.X.X  ( 

6.  TECHNICAL  OFFICE:  ASD/XXPF  ••  •  -  .  .  :  •  .  .  .. 

7.  DO  250  REQ:  LT  ' 

8.  APP  C0DE:_  -  ' 

9.  INPUT  TO  lAC:' ' 

10.  FRSQUENCT;  MTHLT 

11.  AS  OF  OATS:  •  Last  day  of  the  contractor's  monthly  accounting  period. 

12.  DATE  OF  1ST'  SUBMISSION:  Due  NLT  25  calendar  days  after  the  end  of  the 
first  complete  accounting  period  subsequent  to  the  contracting  officer's  . 
authority  to  proceed  (including  undefinitized  contracting  actions).  - 

13.  DATE  OF  SUBSEQUENT  SUBMISSION/EVENT  ID;  Formats  1-4  due  NLT  15  calendar 
days  after  the  end  of  each  monthly  accounting  period.  Format  5  due  NLT  25 
calendar  days  after  the  end  of  each  monthly  accounting  period.  Contractor,  shall 
format  the  data  for  electronic  data  transmission  in  accordance  vith  the  attached 
standard. 

14.  DISTRIBUTION:  ASD/XXXX  ASD/FHCA  AGO 

ASD/XXPF  PCO 

15.  TOTAL: 

16.  REHAJLiCS: 
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SA.ypLZ  COST  PERFORMANCE  REPORT  CTRL 
(COLLAR/ percent  TSRSSHCLCS) 


* 

1. 

SZG0E.MCE  NliM3?-R;  ACOX 

4  . 

TITLE:  COST  PERFOPJiANCE  REPORT  (CPR) 

3. 

SUBTITLE: 

4. 

AUTHORITT:  DI-F-6CO0C/T 

Poraats  1-5  apply.  Contractor  forrat  acceptable  if  it  contains  all  required 
data  and  is  approved  by  the  Government. 

Foraat  1  Contract  Vork  Breakdcvn  Structure  (CVBS)  reporting  eleaents  shall  be  in 
accordance  vith  the  Contractor  Cos:  Data  Reporting  Plan  attached  in  the 
contract  and  shall  include  direct  cost  only.  Overhead/ fri.nge  benefits,  General 
and  Administrative  (GoA),  and  Cos:  of  Money  (COM)  shall  be  reported  as  three 
individual  elements  added  to  the  direct  cost  subtotal.  Hajor/cri tical 
subcontractor  summary  level  performance  measurement  data  shall  be  reported  as  an 
attachment  to  Format  1  (subcontractor  C?R  or  C/SSR  is  acceptable). 

All  Budget  at  Completion  (BAC)  changes  on  Formats  1  and  2  shall  be  explained  in 
Format  5.  The  explanation  of  Management  Reserve  forecasted  consumption  required 
in  Format  5  shall  Include  estimated  dollar  value,  potential  vork  scope,  and 
estimated  budget  distribution  date. 

Reporting  elements  for  For.maCS  2  and ‘4  shall  be  the  second  level  of  the 
contractor's  organizational  structure  (i.e.,  Engineering,  Manufacturing,  etc.). 

Formats  3  and  4  shall  include  identical  forecast  periods.  These  periods  shall 
be  monthly  for  at  least  six  months,  quarterly  for  at'least  tvo  quarters,  and 
then  quarterly,  semi-annually,  or  annually  to  completion.  Any  change  i.-i  the 
Format  3  total  Performance  Measurement  Baseline  (?HB)  and/or  any  change  in  a 
forecast  period  vhich  exceeds  lOZ  shall  be  explained  in  Format  5.  Any  change  in 
current,  forecast  period,  or  total  manpover- for  each  Format  4  element  vhich:. 
exceeds  lOiT  shall  be  explained  in  Format  5.  Include  on  Format’ 4  (as  a  non-add 
item  marked  "BCVS")  the  total  labor  budget  baseline  spread  (in  the  same  units  as 
the  rest  of  Format  4)  extracted  frea  the  total  PMB  dollars  on  Format  3. 

The  Govemaent  reserves  the  right  to  reviev  and  modify,  through  negotiations, 
the  variance  analysis  requirements  for  Formats  1-4  during  the  performance  of  the 
contract,  but  no  sooner  than  six  months  after  contract  avard. 

Variance  analysis  shall  be  reported  for  each  Format  1  and  2  reporting  element 
(including  indirect  cost  elements)  as  follovs  (subcontractor  variance  analysis 
shall  be  included  as  a  part  of  the  prime  contractor's  variance  analysis): 

a.  Current  month  cost  and  schedule  variances  -  plus  or  minus  151  and  $25,CC0. 

b.  Cumulative  cost  and  schedule  variances: 

0-2031  complete  -  plus  or  minus  153^  and  $25,000  (Percent  complete 

equals  cumulative 

2Q-50X  complete  -  plus  or  minus  lOJ  and  $50,000  BCVP  divided  by 

the  PMB)  50-1003: 

complete  -  plus  or  minus  53:  and  $100,000 
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SA.MPL2  COST  PElRFCrwy-^SCZ  REPORT  CCP.L  (CONTINUED) 
(DOLLAa/PZRCZNT  TERZSaOLDS) 


c.  All  variances  at  completion  of  plus  or  minus  5X  and  $50, COO  shall  be 
addressed. 

d.  Specific  corrective  actions,  forecasted  closure  date,  and  impact  to  the 
Estimate  at  Completion  (SAC)  shall  be  included  la  each  variance  narrative. 
Contractor  shall  go  to  the'  loves t  CiTS  level  (including  the  cost  account  level) 
needed  to  completely  explain  the  variances. 

e.  Contractor  shall  format  and  submit  the  CPR  in  accordance  vith  the  attached 
specification  for  electronic  data  transfer. 

5.  CONTPACT  REFSRENCS;  SOV  TASK  2.1.2 


6. 

TECHNICAL  OFFICE: 

ASD/XXPF 

7. 

DD  250  REQ;  LT 

8. 

APP  CODE: 

9. 

I.NPUT  TO  lAC: 

10. 

FREQUENCY:  HTHLY 

11. 

AS  OP  DATE:  Ust 

day  of 

the  contr'actor's  monthly  accounting  period. 

12. 

DATS  OF  1ST  SUBMISSION: 

Due  NLT  25  calendar  days  after  the  end  of  the 

first  complete  accounting  period  subsequent  to  the  contracting  officer's 
authority  to  proceed  (including  undefinitimed  contracting  actions). 

13.  DATS  OF  SUBSEQUENT  SUBMISSI0K/3VENT  ID:  Forets  1-4  due  NLT  15  calendar 
days  after  the  end  of  each  monthly  accounting  period.  Format  5  due  NLT  25 
calendar  days  a5ter  the  end  of  each  monthly  accounting  period.  Contractor,  shall 
format  the.- data- for  electronic  data  tranaalsslou  in  accordanca  vith  the  attached 
standard. 


14. 

DISTRIBUTION: 

ASO/2Z22 

PCO 

« 

•  "  - 

ASD/22PP 

ACO  ASD/FMa 

15. 

TOTAL: 

16. 

REMARKS; 

•  - 

96 


I 


2 , 
4. 


SAMPLE  C0ST/SC3ZEU1*  STATUS  ?S?0 
(TO?  VARZJdiCZ  DRZ'/ZZ.  A?? 

SEQUENCE  NUMBER:  ACOX 

TITLE:  COST/ SCHEDULE  STATUS  REPORT  (C/SSR) 
SUBTITLE: 

AUTHORITY:  Or-T-SOLOA/T 


T  (C/SSR) 
EACH) 


Contract  Vock.  Breakdovn  Structure  (GTBS)  reporting  elements  shall  be  in 
accordance  vith  the  Contractor  Cast  Data  Reporting  Plan  attached  In  the 
contract  and  shall  include  direct  cost  only.  Overhead/fringe  benefits,  General 
and  Adninistrati ve  (GiA),  and  Cost  of  Honey  (COM)  shall  be  reported  as  three 
individual  elements  added  to  the  direct  cost  subtotal.  Major/critical 
subcontractor  sunaary  level  perfarrance  measureaen:  data  shall  be  reported  as  an 
attachment  (subcontractor  C/SSR  is  acceptable).  Contractor  format  acceptable  if 
it  contains  all  required  data  and  is  approved  by  the  Government. 

All  Budget  at  Ccapletion  (BAC)  changes  shall  be  explained  in  the  narrative 
analysis.  The  explanation  of  Management  Reserve  forecasted  cor.suaption  required 
in  the  narrative  analysis  shall  Include  estimated  dollar  value,  potential  vork 
scope,  and  estimated  budget  distribution  dace. 

In  the  narrative  analysis  of  the  first  C/SSR,  the  contractor's  program  manager 
shall  rank.,  in  descending  order  of  criticality  (i.e.,  the  most  critical  element 
at  the  top),  all  C«BS  reporting  ela.ments  anticipated  to  be  schedule  variance 
"drivers."  The  contractor  shall  also  rank.,  in  descending  order  of  criticality, 
all  CV3S  elements  anticipated  to  be  cost  variance  drivers.  The  Government 
reserves  the  right  to  modify  these  ra.-Jcings  based  on  its  perception  of 
criticality.  .  Contractor  shall  submit  updated  schedule  variance  and  cost 
variance  rankings  every  six  months,  as  a  minimum,  based  on  performance  to  date 
and  anticipated  problems.  If  Che  contractor  uses  critical  pat.h/netvorking 
schedule  techniques,  identification  of  the  critical  path  and  near  critical  path 
CTBS  elements  shall  meet  the  schedule  driver  requirement. 

Variance  analysis  shall  be  reported  for  each  reporting  element  as  follovs 
(subcontractor  variance  analysis  shall  be  included  as  a  part  of  the  prim* 
contractor's  variance  analysis): 

a.  Cumulative  schedule  variances  (S7)  -  SV  narratives  shall  be  submitted  for 
those  CVBS  elements  vhich  represent  t.he  top  SOT  of  the  SV  drivers  discussed 
above,  vith  a  minimum  SV  of  $25,000.  If  there  are  fever  than  10  schedule 
variance  drivers,  all  variances  over  $25,000  shall  be  explained. 

b.  Cumulative  cost  variances  ((7/)  -  CV  narratives  shall  be  submitted  for  those 
CT3S  elements  vhich  represent  the  top  50Z  of  the  CV  drivers  discussed  above, 
vith  a  minimum  (7/  of  $25,000.  If  there  are  fever  than  10  cost  variance  drivers, 
all  variances  over  $25,000  shall  be  explained. 

c.  All  cumulative  cost  and  schedule  variances  for  the  indirect  cost  elements 
vhich  exceed  -*■/-  lOT  and  $25,000  shall  be  explained. 

d.  All  variances  at  completion  of  plus  or  minus  5T  and  $25,000  shall  be 
addressed. 
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SAHPL2  C0ST/SCHZ2ULZ  STATUS  REPORT  (C/SSR)  CTRL  (CUNTIItUiD) 
(TOP  .ar:a.'*c2  driver  A???.GACH) 


e.  Specific  correccive  actions,  forecasted  closure  date,  and  iapact  to  the 
Estiaate  at  Completion  (EAC)  shall  be  included  in  each  variance  narrative. 
Contractor  shal^  go  to  the  lovest  C’.'ES  level  needed  to  completely  explain  the 
variances • 

5.  CONTR-ACT  RSFEIRSUCE:  SOW  TASR  l.I.l 

6.  TECEHIC.AL  OFFICE:  ASD/T2?? 

7.  DD  250  R2Q;  LT 

8.  AP?  CODS: 


9.  INPUT  70  lAC: 

10.  FREQUENCY:  MT3LT 

11.  AS  OP  DATS:  Last  day  of  the  contractor'^  monthly  accounting  period. 

12.  DATE  0?  1ST  SUBMISSION:  Due  ^fLT  25  calendar  days  after  the  end  of  the 
first  complete  accounting  period  subsequent  to  the  contracting  officer's 
authority  to  proceed  (including  undef ini  tired  contracting  actions). 

13.  DATS  0?  SUBSEQUENT  SU3HISSI0N/EVENT  ID:  C.'BS  format  due  NLT  15  calendar 
days  after  the  end  of  each  nonthly  accounting  period.  Narrative  analysis  due 
NLT  25  calendar  days  after  the  end  of  each  monthly  accounting  period. 
Contractor  shall  format  the  data  for  electronic  data  transmission  in  accordanc 


vith 

the  attached 

standard. 

14. 

DISTRIBUTION; 

ASD/XZZI  - 

PCO 

ASD/XX?? 

ASD/FMCA 

ACO 

15. 

TOTAL; 

Id'. 

REMARKS: 
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S.\.M?LZ  COST/SCH£CL"Z  STATUS  ££?ORT  (C/SSR)  CTRL 
(T0LL.AR/?i?.C2:nr  TS12SH0LTS) 

1.  SZCl'I.NCZ  N'UHZZ?.;  ACCX 

2.  TITLZ;  COST/SC'dZZULZ  STATUS  SZPCRT  (C/SSR) 

3.  3U3TITL2: 

4.  AUTHCRITT:  DI-F-6Q1QA/T 

Contract  Toric  Zrsakdovn  Structure  (CoS)  reportinj  eleaents  shall  be  in 
accordance  vith  the  Contractor  Cos:  Tata  Reporting  Plan  attached  in  the 
contract  and  shall  include  direct  cos:  only.  Overhead/fringe  benefits,  General 
and  Adainistrative  (G4A) ,  and  Cost  o:  Money  (COM)  shall  be  reported  as  three 
individual  eleaents  added  to  the  direct  coat  subtotal.  Hajor/critical 
subcontractor  suaaary  level  perforrance  aeasureaent  data  shall  be  reported  as  an 
attachaent  (subcontractor  C/SSR  is  acceptable).  Contractor  foraat  acceptable  if 
it  contains  all  required  data  and  is  approved  by  the  Government. 

All  Budget  at  Completion  (3AC)  changes  shall  be  explained  in  the  narrative 
analysis.  The  explanation  of  Managecent  Reserve  forecasted  consuaption  required 
in  the  narrative  analysis  shall  include  estimated  dollar  value,  potential  vcrx 
scope,  and  estimated  budget  distribution  date. 

The  Government  reserves  the  right  to  reviev  and  modify,  through  negotiations, 
the  variance  analysis  requirements  during  the  performance  of  the  contract,  but 
no  sooner  than  six  months  after  contract  avard. 

Variance  analysis  shall  be  reported  for  each  reporting  element  (including 
indirect  cost  elements)  as  follovs  (subcontractor  variance  analysis  shall  be 
included  as  a  part  of  the  prime  contractor's  variance  analysis): 

a.  Cumulative  cost  and  schedule  variances: 

0-202  complete  -  plus  or  minus  lf2  and  $10,000  (Percent  complete 

equals  cumulative 

20-502  complete  -  plus  or  minus  102  and  $25,000  BCV?  divided  by  the 

Performance  Measure- 

50-1002  complete  -  plus  or  minus  52  and  $50,000  ment  Baseline) 

b.  All  variances  at  completion  of  plus  or  minus  52  and  $25,000  shall  be 
addressed. 

c.  Specific  corrective  actions,  forecasted  closure  date,  and  impact  to  the 
Estimate  at  Completion  (EAC)  shall  be  included  in  each  variance  narrative. 
Contractor  shall  go  to  the  loves t  CV3S  level  needed  to  completely  explain  the 
variances. 

5.  COffTRACT  REFERZNCS:  SOV  TASK  X.X..T 


6.  TEC.HJJICAL  OFrIC: 


ASD/:a?F 


SAMPL2  C0ST/SCSEDUL3  STATUS  REPORT  (C/SSR)  CDRL  (CONTINUED) 
(DOLLAR/ PERCENT  THRESHOLDS) 


7. 

DD  2f0  RSQ: 

LX 

8. 

APP  CODE: 

9. 

INPUT  TO  lAC: 

10. 

FREQUSNCT: 

HTHLY 

11. 

AS  OP  DATS: 

Last  day  of  the  contractor's  monthly  accounting  period. 

12. 

DATS  OP  1ST 

SUBMISSION:  Due  HLT  25  calendar,  days  after  tha-end  of  the 

first  complete  accounting  period  subsequent  to  the  contracting  officer's 
authority  to  proceed  (including  undefinitized  contracting  actions). 

13.  DATE  OP  SUBSEQUENT  SUBHISSIOH/EVENT  ID:  CTBS  format  due  NLT  15  calendar 
days  after  the  end  of  each  monthly  accounting  period.  Narrative  analysis  due 
NLT  25  calendar  days  after  the  end  of  each  monthly  accounting  period. 

Contractor  shall  format  the  data  for  electronic  data  transmission  in  accordance 
with  the  attached  standard. 


14. 

DISTRIBUTION: 

ASD/XXXX 

ASD/XXPP 

ASD/FMCA 

PCO 

ACO 

15. 

TOTAL:  . 

16. 

REMARKS: 

Defense  Contractor  A 


Performance  Measurement  System  Review 

-  Gave  information  on  when  variance  analysis  reports  must  be  generated  and  who  is 
responsible  for  setting  cost  account  variance  analysis  thresholds/parameters  (p  4  of  7). 

-  No  specifics  on  how  the  variance  thresholds  were  established. 


Defense  Contractor  B 

-  Tolerances  are  expressed  as  being  plus  or  minus  a  percentage  and  a  minimum  value; 
e.g.  +/-  10  percent  and  $10,000  for  nonlabor,  and  +/-  10  percent  and  300  hours  for 
labor.’ 

-  ’They  are  applied  to  cumulative-to-date  variance  at  two  levels:  the  cost  account  level 
and  the  report  level.  They  can  be  different  at  each  level  and  can  vary  from  program 
to  program.  Cost  account  tolerances  are  established  by  the  program.  ’ 


Defense  Contractor  C 

-  ’If  the  tolerances  are  exceeded  either  plus  or  minus,  the  variances  are  flagged  for 
special  analysis  as  a  problem  area.’ 

-  ’Responsibilities’ 

—  Cost  Control  -  Establish  all  tolerance  bands  for  variance  analysis.’ 


Defense  Contractor  D 

-  Responsibilities  for  Cost  Management 

—  Establish  all  tolerance  bands  for  variance  analysis. 

-  Tolerances 

—  Exactly  the  same  as  March  19xx  vereion 
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Defense  Contractor  E 


-  ’Thresholds  are  established  which  set  parameters  for  variance  analysis  reporting.’ 

'  ’Normally  the  contract  sets  forth  the  thresholds  for  external  reporting’ 

-  ’If  no  reporting  thresholds  are  established  in  the  contract,  the  Program  Manager  will 
still  establish  variance  analysis  thresholds.’ 


Defense  Contractor  F 

-  ’Variance  threshold  levels  above  which  analysis  is  required  as  specified  in’  another 
document. 

-  This  document  ’identifies  instructions  applicable  to  C/SCS  programs.  Variance 
thresholds,  high-cost  material  definition,  CPR  reponing  levels,  and  other  requirements 
are  discussed.’ 


Defense  Contractor  G 

-  ’Normally  contractually  established  for  each  program’ 

-  ’They  may  be  reviewed  and  changed  during  the  program  if  necessary.’ 


Defense  Contractor  H 

-  ’Tolerances  always  will  be  established  for  cum-to-date  and  at-completion  variance 
and  may  be  established  for  current  period  variances.  Also  the  tolerances  could  contain 
a  minimum  value;  for  example,  20%  or  $50,000,  which  is  less,  favorable  or 
unfavorable,  but  at  least  $15,000.’ 

-  ’Other  combinations  of  tolerances  are  possible  and  depend  on  what  the  customer 
believes  will  best  serve  his  needs.’ 


Defense  Contractor  I 

-  In  the  appendix  of  document,  it  de.scribed  variance  thresholds  as  a  percentage  and/or 
an  established  hour  or  dollar  minimum.  No  specific  method  for  establishing  variance 
thresholds  was  mentioned. 
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Appendix  E:  US.  Army  Study 
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ESTABLISHMENT  OF 
CONSISTENT  VARIANCE  THRESHOLDS 
FOR  PROBLEM  ANALYSIS 
ON  THE 

COST  PERFORMANCE  REPORT 
FOR  THE 

Load  PPD  program  PHASE  13 


30  JUNE  1981 


PREPARED  BY 


TSLEDYNE  BROWN  ENGINEERING 
RESEARCH  PARK 
HUNTSVILL'. ,  ALABAMA  35807 


PREPARED  FOF 


BALLISTIC  MISSILE  DEFENSE  SYSTEMS  COMMAND 
SYSTEMS  TECHNOLOGY  PROGRAM 
LOW  ALTITUDE  DEFENSE  PREPROTOTYPE  PROGRAM 
HUNTSVILLE,  ALABAMA  35806 
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3C  JUNE  1981 

The  attached  paper  presents  a  sethcd  of  calcclatir.g  cost  and 
schedule  thresholds  for  variances  cr.  the  cost  perfcrnance  report. 

The  equation  shown  in  the  attached  paper  is  being  considered  for  incor¬ 
poration  in  TS  705-622.i\  in  place  of  the  table  shown  on  page  9  (Table 
1).  Parag.“aph  3-3-3-1  of  TS  705-622A  would  be  c.hanged  to  delete  the 
_ney t-to-last  sentence  in  the  paragraph,  which  refers  to  Table  1.  A 
set  of  words  siailar  to  the  follcwirg  wculd  beccne  part  of  pai-agraph 

3.3.3. 1. 

Words  for  TS  705-622 

Page  7,  paragraph  2,  seventh  sentence  -  delete.  Replace  with: 

"For  completicn  type  contracts,  the  thresholds  for  eyplaini.ng 
variances  shall  be  coaputed  monthly  by  the  equation  shown  below  unless 
otherwise  specified  in  the  contract. 


where 


BAG  r  Contract  budget  at  completion 

BCW?  =  The  cumulative  budgeted  cost  of  work  performed 

for  the  cost  elesent  involved  through  the  end  of 
the  reporting  period. 

These  thresholds  shall  be  applied  routinely  at  level  three  and  above 
of  the  WES.  The  application  of  the  equation  to  lower-level  WBS  elements 
shall  be  based  on  specific  Government  management  needs  for  limited 
time  periods.  The  Government  reserves  the  right  to  require  C?R  reporting 
at  lower  level  of  the  WBS  when  significant  budget/management  needs 
are  identified,  or  when  specified  in  the  contract  in  accordance  .with 
the  contract  DO  Form  1423-  The  evpar.ded  reporting  prerogative  will 
be  utilised  to  the  minimum  extent  consistent  with  sou.nd  management 
control." 
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CCNSISTIMT  VAP.lAfICZ  THHEcHCL-S 


[A2  presently  formulated  the  LoAD  variance  thresholds  are  diaccn- 
tinuous  frca  year  to  year-  Ihis  leads  to  a  situation  where  a  given 
variance  on  a  cost  reporting  elenent  cay  be  within  threshold  at  the 
end  of  one  year  and  beyond  threshold  at  the  beginning  cf  the  next. 

In  addition,  the.''e  were  both  percentage  and  afascluce  thresholds  at 
level  three  of  the  'ticS,  For  sufficiently  large  elenar.os,  one  absolute 
values  are  exceeded  before  the  percentage  t.hreshclds;  for  small  elements, 
the  obverse  is  true.  Lastly,  the  t.hreshclds  va.-y  with  tine  in  such 
a  oa.nner  that  the  magnitude  of  the  elecent  existi-ng  at  the  crossover 
froQ  percentage  to  absolute  varies  tremendously  wioh  time. 

This  memo  prsse.nts  a  means  cf  settng  thresholds  which  does 
not  suffer  from  the  above  inconsistencies  and  irrationalities.  In 
the  following  derivation,  it  is  assumed  that  variable  thresholds  are 
the  result  of  an  attempt  to  take  account  of  the  statistics  of  estimating 
error.  Let  us  consider  this  in  terms  of  the  standard  deviation  of 
the  estimate,  o  .  In  accordance  with  the  requirements  of  C/SCSC,  an 
estimate  or  budgeted  cost  of  work  to  be  performed  at  a.ny  level  of  the 
WBS  is  the  sunmation  of  a  number  of  such  estimates  at  the  lowest  level: 
For  the  moment,  we  will  assume  that  the  lowest  level  estimates  are 
roughly  equal  in  magnitude  and-  have  the  same  standard  deviation.  Given 
this,  the  standard  deviation  as  a  percentage  cf  the  estimate  will  become 
smalle."  as  one  moves  up  the  WBS.  It  further  follows  from  the  assumption 

/ 

of  equal  magnitude  and  o  's  of  the  elemental  costs  that  the  standard 
deviation  in  percentage  terms  of  any  work  element  at  any  level  in  the 
WHS  is  related  to  that  for  any  other  element  as  the  squa.^e  root  of 
the  inverse  ratio  of  the  two  costs  per  the  following. 

2 

The  square  of  the  standard  deviation  at  one  level,  ,  is 

euql  to  the  sum  of  os  at  the  lower  level 


2  X  2 
°1  ""2  ^ 


•  o 
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If,  aa  r.otad  above,  ail  low  level  os  are  equal. 


c  “ 

or 

2  2 

Cl  -  <^3 

2  2 

^2  "  “2  °3 

Also  the  tota.1  coat  for  any  elenenc,  C^,  is  equal  to  the  sun  of  the 
costs  of  the  work  packages  comprisisg  it-  Since  these  are  equal; 

Froa  the  above 


S  itapl  I  fy  Ing. 


where 

CT  *  Standard  deviation  of  esciaate  for  eienent  n 
n 

C  =  ciraulative  cost  estirata  for  eieaesc  n 
n 

Cg  =  cost  of  lowest  level  elaaent 

<7g  *  standard  deviation  of  error  in  estinate  at  lowest  level 

la  the  above  the  cost  estimate  (C^)  may  be  for  the  total  for  the 
work  eleaent,  or  the  total  for  any  given  period  of  tiae.  Thus,  the  large 
Che  cost  of  an  eleaent,  the  lower  the  percentage  error  in  the  estiaate  of 
that  cost;  and,  the  longer  work  has  progressed  (and  the  more  spent  on  it) 
Che  saaller  the  percentage  error  of  the  estiaate. 

Now 
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1/2 


If  we  assume  a  perceniage  variance  allcwable  ac  camplecicn 

and  sec  Ic  ac  X,  che  allowable  variation  ac  any  level  for  any  work  element 
would  be 


Since  -pr”  is  a  percentage,  che  absolute  wcult  be 


1/2 


m 


\jC^j  ^1  *"L  cumulative  cost  o:  work  perfome^CBC^?)  , 

for  any  elemenc  ac  any  time  and  is  che  concracc  budgeted  co*sc  ' 
ac  complecion,  che  variance  for  any  element  ac  any  cine  becoces 
(aOT)"-'^  (3AC)  •  X.  If  che  program  office  secs  a  value  for  ’ 

X*  all  variances  follow.  The  above  equation  may  be  rescaced  as 


•  X<^-^ 
^  I  BCWP 


1/2 


(3C<7) 


Since  X 


BAG 


1/2 


J  a  percentage,  this  in  effect  says  that  che  percenCags 

variance^allowable  for  any  cost  element  is  larger  than  chat  allowable  for  * 

.1/2 

che  contract  budget  at  complecion  by  che  factor 


w 


One  can  also  empress  che  relationship  as 


(X  •  BAG) 


Bq? 

BAG 


1/2 
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This  iii.iv  1h-  1  n  C  f  r ; '  r  I- 1  fd  lo  nu-.Tn  'ii,'.:  .-sijso  !  ul  i-  v.i  r  i  .'.nc  a  !  ;  I'v.'.b  1  for 

any  cusc  elcnu-nc  Is  saialler  than  tiia:  allowable;  fcr  budg-:  at  Ion 

(X  ‘  EaC)  by  Che  faccar 

Ic  was  iniclally  assuisec  for  che  purposes  of  che  analysis  chac 
Che  coses  of  Lowesc  level  work  package  were  of  equal  Tnagnicude  and  had 
equal  uncercaincies  associaced  wlch  cheir  cosc  esrinsaces.  Ic  say  be 
objecced  chac  chese  assuiapcions  are  so  rescrlccive  as  co  never  be  mec  and 
cherefore  chac  che  mechodology  canne:  be  used.  In  face  che  asstrrpcior.s 
need  noC  be  so  rescrlccive,  Ic  can  be  proven  chac  che  conciusiens  are 
valid  if  che  relacionship 


holds;  chac  is,  if  che  ratio  of  che  work  package  variance  is  proporcional 
Co  che  inverse  square  root  of  Che  ratio  of  che  coses  of  che  work  packages. 
They  are  also  valid  If  che  standard  deviations  (<7s)  are  equal  for  work  ^ 
packages  of  equal  magnitude  and  each  work  element  at  che  high  levels  of  che 
WBS  comprises  equally  proportioned  nixes  ox  different  sized  work  packages. 

These  may  yec  seem  racher  restrictive.  However  Che  averaging 
process  as  one  moves  up  che  VHS  is  such  that  only  approximate  adherance  Co 
these  conditions  will  result  in  the  conclusions  being  quite  accurate. 

A  comparison  of  che  application  of  this  approach  to  che  prior  one 
may  be  seen-ia  Figure  1.  This  shews  the  level  three  breakout  of  a  level 
two  work  element,  the  L.I.S.  The  CuH  $  figures  in  each  block  represent 
che  cumulative  year  by  year  cosc  estimates.  They  may  be  considered  the 
cumulative  BCWP,  at  the  ead  of  each  year.  The  old  figures  are  che  variance 
reporting  thresholds  per  the  current  issue  of  TS  705-622,  The  new  figures 
are  the  thresholds  as  computed  from  the  equation  derived  above.  It  may  be 
seen  that  for  small  cumulative  BCW?  figures,  che  new  thresholds  are  usually 
lower;  for  large  values  of  BCVT?,  the  old  are  lover. 


3CV? 

BAG 


1/: 
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There  are  various  advantages  to  the  previous  way  of  setting 
variance  thresholds,  chiefly: 

1)  There  are  no  discontinuities  in  threshold  year  end  to  year 
beginning . 

2)  The  threshold  for  any  element  of  work  for  any  period  of  time 
is  related  to  all  others  via  the  statistics  of  the  estimating  errors. 

3)  No  tabular  threshold  dada  need  be  entered  into  a  computer. 

One  simple  equation  is  ail  that  is  needed  for  all  threshold 
computations . 

There  is  one  c.hange  in  costi.ng  practice  which  is  required  for  the 
method  to  work.  The  costs  used  must  be  the  mean  costs,  not  the  most 
likely  ones.  The  tow  are  different  since  Che  estimating  error  has  a 
skewed  distribution.  (An  estimated  cost  can  be  overrun  more  than  it  can 
be  under  run) .  This  may  be  an  advantage  in  terms  of  forcing  more 
realistic  program  cost  estimation.  The  expected  value  or  mean  cost  for 
the  program  is  the  sum  of  the  means  of  elemental  cost  estimates  and  with 
a  distribution  skewed  as  mentioned  above  the  sum  of  most  likely  costs  is 
less  than  the  sum  of  the  means.  Thus,  insofar  as  a  program  cost 
estimate  is  the  sum  of  the  most  likely  costs  of  its  elements,  it  will  be 
an  underestimate. 
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